diff --git a/docs/concepts/databases/mongodb.md b/docs/concepts/databases/mongodb.md index 85cd33d23..d36c9b3ec 100644 --- a/docs/concepts/databases/mongodb.md +++ b/docs/concepts/databases/mongodb.md @@ -36,7 +36,7 @@ spec: requests: storage: 50Mi databaseSecret: - secretName: mgo1-admin-auth + secretName: mgo1-auth nodeSelector: disktype: ssd init: @@ -59,7 +59,7 @@ spec: cpu: "500m" doNotPause: true monitor: - agent: coreos-prometheus-operator + agent: prometheus.io/coreos-operator prometheus: namespace: demo labels: @@ -81,22 +81,33 @@ spec: ### spec.storage `spec.storage` is an optional field that specifies the StorageClass of PVCs dynamically allocated to store data for the database. This storage spec will be passed to the StatefulSet created by KubeDB operator to run database pods. You can specify any StorageClass available in your cluster with appropriate resource requests. If no storage spec is given, an `emptyDir` is used. - - `spec.storage.storageClassName` is the name of the StorageClass used to provision PVCs. PVCs don’t necessarily have to request a class. A PVC with its storageClassName set equal to "" is always interpreted to be requesting a PV with no class, so it can only be bound to PVs with no class (no annotation or one set equal to ""). A PVC with no storageClassName is not quite the same and is treated differently by the cluster depending on whether the DefaultStorageClass admission plugin is turned on. +- `spec.storage.storageClassName` is the name of the StorageClass used to provision PVCs. PVCs don’t necessarily have to request a class. A PVC with its storageClassName set equal to "" is always interpreted to be requesting a PV with no class, so it can only be bound to PVs with no class (no annotation or one set equal to ""). A PVC with no storageClassName is not quite the same and is treated differently by the cluster depending on whether the DefaultStorageClass admission plugin is turned on. - - `spec.storage.accessModes` uses the same conventions as Kubernetes PVCs when requesting storage with specific access modes. +- `spec.storage.accessModes` uses the same conventions as Kubernetes PVCs when requesting storage with specific access modes. - - `spec.storage.resources` can be used to request specific quantities of storage. This follows the same resource model used by PVCs. +- `spec.storage.resources` can be used to request specific quantities of storage. This follows the same resource model used by PVCs. To learn how to configure `spec.storage`, please visit the links below: - - https://kubernetes.io/docs/concepts/storage/persistent-volumes/#persistentvolumeclaims + +- https://kubernetes.io/docs/concepts/storage/persistent-volumes/#persistentvolumeclaims ### spec.databaseSecret -`spec.databaseSecret` is an optional field that points to a Secret used to hold credentials for `mongodb` super user. If not set, KubeDB operator creates a new Secret `{mongodb-object-name}-admin-auth` for storing the password for `mongodb` superuser for each MongoDB object. If you want to use an existing secret please specify that when creating the MongoDB object using `spec.databaseSecret.secretName`. +`spec.databaseSecret` is an optional field that points to a Secret used to hold credentials for `mongodb` super user. If not set, KubeDB operator creates a new Secret `{mongodb-object-name}-auth` for storing the password for `mongodb` superuser for each MongoDB object. If you want to use an existing secret please specify that when creating the MongoDB object using `spec.databaseSecret.secretName`. -This secret contains a `.admin` key which contains the password for `mongodb` superuser. Example: +This secret contains a `user` key and a `password` key which contains the `username` and `password` respectively for `mongodb` superuser. Example: ```ini -vPlT2PzewCaC3XZP +apiVersion: v1 +data: + password: NnE4dV8yak1PVy1PT1pYaw== + user: cm9vdA== +kind: Secret +metadata: + ... + name: mgo1-auth + namespace: demo + ... +type: Opaque ``` @@ -163,17 +174,21 @@ KubeDB supports taking periodic snapshots for MongoDB database. This is an optio ### spec.doNotPause `spec.doNotPause` is an optional field that tells KubeDB operator that if this MongoDB object is deleted, whether it should be reverted automatically. This should be set to `true` for production databases to avoid accidental deletion. If not set or set to false, deleting a MongoDB object put the database into a dormant state. THe StatefulSet for a DormantDatabase is deleted but the underlying PVCs are left intact. This allows user to resume the database later. +### spec.imagePullSecret +`KubeDB` provides the flexibility of deploying MongoDB database from a Private Docker Registry. To learn how to deploym MongoDB from a Private Registry, please visit [here](/docs/guides/mongodb/private-registry/using-private-registry.md). ### spec.monitor -To learn how to monitor MongoDB databases, please visit [here](/docs/concepts/monitoring.md). +MongoDB managed by KubeDB can be monitored with builtin-Prometheus and CoreOS-Prometheus operator out-of-the-box. To learn more, +- [Monitor MongoDB with builtin Prometheus](/docs/guides/mongodb/monitoring/using-builtin-prometheus.md) +- [Monitor MongoDB with CoreOS Prometheus operator](/docs/guides/mongodb/monitoring/using-coreos-prometheus-operator.md) ### spec.resources `spec.resources` is an optional field. This can be used to request compute resources required by the database pods. To learn more, visit [here](http://kubernetes.io/docs/user-guide/compute-resources/). ## Next Steps -- Learn how to use KubeDB to run a MongoDB database [here](/docs/guides/mongodb/overview.md). +- Learn how to use KubeDB to run a MongoDB database [here](/docs/guides/mongodb/README.md). - See the list of supported storage providers for snapshots [here](/docs/concepts/snapshot.md). - Thinking about monitoring your database? KubeDB works [out-of-the-box with Prometheus](/docs/guides/monitoring.md). - Learn how to use KubeDB in a [RBAC](/docs/guides/rbac.md) enabled cluster. diff --git a/docs/examples/mongodb/demo-1.yaml b/docs/examples/mongodb/Initialization/demo-1.yaml similarity index 94% rename from docs/examples/mongodb/demo-1.yaml rename to docs/examples/mongodb/Initialization/demo-1.yaml index 9862be5f5..1d68762f1 100644 --- a/docs/examples/mongodb/demo-1.yaml +++ b/docs/examples/mongodb/Initialization/demo-1.yaml @@ -1,7 +1,7 @@ apiVersion: kubedb.com/v1alpha1 kind: MongoDB metadata: - name: mgo1 + name: mgo-init-script namespace: demo spec: version: 3.4 diff --git a/docs/examples/mongodb/Initialization/demo-2.yaml b/docs/examples/mongodb/Initialization/demo-2.yaml new file mode 100644 index 000000000..7c8dbf61f --- /dev/null +++ b/docs/examples/mongodb/Initialization/demo-2.yaml @@ -0,0 +1,18 @@ +apiVersion: kubedb.com/v1alpha1 +kind: MongoDB +metadata: + name: mgo-init-snapshot + namespace: demo +spec: + version: 3.4 + storage: + storageClassName: "standard" + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 50Mi + init: + snapshotSource: + name: snapshot-infant + namespace: demo diff --git a/docs/examples/mongodb/monitoring/builtin-prometheus/demo-1.yaml b/docs/examples/mongodb/monitoring/builtin-prometheus/demo-1.yaml new file mode 100644 index 000000000..14caaa075 --- /dev/null +++ b/docs/examples/mongodb/monitoring/builtin-prometheus/demo-1.yaml @@ -0,0 +1,16 @@ +apiVersion: kubedb.com/v1alpha1 +kind: MongoDB +metadata: + name: mgo-mon-prometheus + namespace: demo +spec: + version: 3.4 + storage: + storageClassName: "standard" + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 50Mi + monitor: + agent: prometheus.io/builtin diff --git a/docs/examples/monitoring/demo-2.yaml b/docs/examples/mongodb/monitoring/coreos-operator/demo-1.yaml similarity index 76% rename from docs/examples/monitoring/demo-2.yaml rename to docs/examples/mongodb/monitoring/coreos-operator/demo-1.yaml index e17fc03f3..51ddd65b7 100644 --- a/docs/examples/monitoring/demo-2.yaml +++ b/docs/examples/mongodb/monitoring/coreos-operator/demo-1.yaml @@ -1,10 +1,10 @@ apiVersion: kubedb.com/v1alpha1 -kind: Postgres +kind: MongoDB metadata: - name: pmon + name: mgo-mon-coreos namespace: demo spec: - version: 9.5 + version: 3.4 storage: storageClassName: "standard" accessModes: @@ -13,9 +13,10 @@ spec: requests: storage: 50Mi monitor: - agent: coreos-prometheus-operator + agent: prometheus.io/coreos-operator prometheus: namespace: demo labels: app: kubedb interval: 10s + diff --git a/docs/examples/mongodb/private-registry/demo-1.yaml b/docs/examples/mongodb/private-registry/demo-1.yaml new file mode 100644 index 000000000..eafdf211f --- /dev/null +++ b/docs/examples/mongodb/private-registry/demo-1.yaml @@ -0,0 +1,8 @@ +apiVersion: v1 +kind: Secret +metadata: + name: myregistrykey + namespace: demo +data: + .dockerconfigjson: +type: kubernetes.io/dockerconfigjson diff --git a/docs/examples/mongodb/private-registry/demo-2.yaml b/docs/examples/mongodb/private-registry/demo-2.yaml new file mode 100644 index 000000000..a810c67c8 --- /dev/null +++ b/docs/examples/mongodb/private-registry/demo-2.yaml @@ -0,0 +1,17 @@ +apiVersion: kubedb.com/v1alpha1 +kind: MongoDB +metadata: + name: mgo-pvt-reg + namespace: demo +spec: + version: 3.4 + doNotPause: true + storage: + storageClassName: "standard" + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 50Mi + imagePullSecrets: + - name: myregistrykey diff --git a/docs/examples/mongodb/quickstart/demo-1.yaml b/docs/examples/mongodb/quickstart/demo-1.yaml new file mode 100644 index 000000000..509737fd3 --- /dev/null +++ b/docs/examples/mongodb/quickstart/demo-1.yaml @@ -0,0 +1,15 @@ +apiVersion: kubedb.com/v1alpha1 +kind: MongoDB +metadata: + name: mgo-quickstart + namespace: demo +spec: + version: 3.4 + doNotPause: true + storage: + storageClassName: "standard" + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 50Mi diff --git a/docs/examples/mongodb/snapshot/demo-1.yaml b/docs/examples/mongodb/snapshot/demo-1.yaml new file mode 100644 index 000000000..cbe1bb9a2 --- /dev/null +++ b/docs/examples/mongodb/snapshot/demo-1.yaml @@ -0,0 +1,15 @@ +apiVersion: kubedb.com/v1alpha1 +kind: MongoDB +metadata: + name: mgo-infant + namespace: demo +spec: + version: 3.4 + doNotPause: true + storage: + storageClassName: "standard" + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 50Mi diff --git a/docs/examples/mongodb/demo-2.yaml b/docs/examples/mongodb/snapshot/demo-2.yaml similarity index 78% rename from docs/examples/mongodb/demo-2.yaml rename to docs/examples/mongodb/snapshot/demo-2.yaml index dd490b072..42d92be3a 100644 --- a/docs/examples/mongodb/demo-2.yaml +++ b/docs/examples/mongodb/snapshot/demo-2.yaml @@ -1,12 +1,12 @@ apiVersion: kubedb.com/v1alpha1 kind: Snapshot metadata: - name: mgo-xyz + name: snapshot-infant namespace: demo labels: kubedb.com/kind: MongoDB spec: - databaseName: mgo1 + databaseName: mgo-infant storageSecretName: mg-snap-secret gcs: bucket: restic diff --git a/docs/examples/mongodb/demo-4.yaml b/docs/examples/mongodb/snapshot/demo-3.yaml similarity index 78% rename from docs/examples/mongodb/demo-4.yaml rename to docs/examples/mongodb/snapshot/demo-3.yaml index 4e87a3607..79ba20b8f 100644 --- a/docs/examples/mongodb/demo-4.yaml +++ b/docs/examples/mongodb/snapshot/demo-3.yaml @@ -1,7 +1,7 @@ apiVersion: kubedb.com/v1alpha1 kind: MongoDB metadata: - name: recovered + name: mgo-recovered namespace: demo spec: version: 3.4 @@ -14,4 +14,5 @@ spec: storage: 50Mi init: snapshotSource: - name: mgo-xyz + name: snapshot-infant + namespace: demo diff --git a/docs/examples/mongodb/demo-3.yaml b/docs/examples/mongodb/snapshot/demo-4.yaml similarity index 95% rename from docs/examples/mongodb/demo-3.yaml rename to docs/examples/mongodb/snapshot/demo-4.yaml index fc2ed1971..281950650 100644 --- a/docs/examples/mongodb/demo-3.yaml +++ b/docs/examples/mongodb/snapshot/demo-4.yaml @@ -1,7 +1,7 @@ apiVersion: kubedb.com/v1alpha1 kind: MongoDB metadata: - name: mgo1 + name: mgo-scheduled namespace: demo spec: version: 3.4 diff --git a/docs/examples/monitoring/builtin-prometheus/demo-0.yaml b/docs/examples/monitoring/builtin-prometheus/demo-0.yaml new file mode 100644 index 000000000..1d9f59eb3 --- /dev/null +++ b/docs/examples/monitoring/builtin-prometheus/demo-0.yaml @@ -0,0 +1,7 @@ +apiVersion: v1 +kind: Namespace +metadata: + name: demo +spec: + finalizers: + - kubernetes diff --git a/docs/examples/monitoring/builtin-prometheus/demo-1.yaml b/docs/examples/monitoring/builtin-prometheus/demo-1.yaml new file mode 100644 index 000000000..8803354f8 --- /dev/null +++ b/docs/examples/monitoring/builtin-prometheus/demo-1.yaml @@ -0,0 +1,43 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: prometheus-server-conf + labels: + name: prometheus-server-conf + namespace: demo +data: + prometheus.yml: |- + global: + scrape_interval: 5s + evaluation_interval: 5s + scrape_configs: + - job_name: 'kubernetes-service-endpoints' + + kubernetes_sd_configs: + - role: endpoints + + relabel_configs: + - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scrape] + action: keep + regex: true + - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scheme] + action: replace + target_label: __scheme__ + regex: (https?) + - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_path] + action: replace + target_label: __metrics_path__ + regex: (.+) + - source_labels: [__address__, __meta_kubernetes_service_annotation_prometheus_io_port] + action: replace + target_label: __address__ + regex: ([^:]+)(?::\d+)?;(\d+) + replacement: $1:$2 + - action: labelmap + regex: __meta_kubernetes_service_label_(.+) + - source_labels: [__meta_kubernetes_namespace] + action: replace + target_label: kubernetes_namespace + - source_labels: [__meta_kubernetes_service_name] + action: replace + target_label: kubernetes_name diff --git a/docs/examples/monitoring/builtin-prometheus/demo-2.yaml b/docs/examples/monitoring/builtin-prometheus/demo-2.yaml new file mode 100644 index 000000000..ffd6b710c --- /dev/null +++ b/docs/examples/monitoring/builtin-prometheus/demo-2.yaml @@ -0,0 +1,51 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: prometheus-server + namespace: demo +spec: + replicas: 1 + selector: + matchLabels: + app: prometheus-server + template: + metadata: + labels: + app: prometheus-server + spec: + containers: + - name: prometheus + image: prom/prometheus:v2.1.0 + args: + - "--config.file=/etc/prometheus/prometheus.yml" + - "--storage.tsdb.path=/prometheus/" + ports: + - containerPort: 9090 + volumeMounts: + - name: prometheus-config-volume + mountPath: /etc/prometheus/ + - name: prometheus-storage-volume + mountPath: /prometheus/ + volumes: + - name: prometheus-config-volume + configMap: + defaultMode: 420 + name: prometheus-server-conf + - name: prometheus-storage-volume + emptyDir: {} + +--- +apiVersion: v1 +kind: Service +metadata: + name: prometheus-service + namespace: demo +spec: + selector: + app: prometheus-server + type: LoadBalancer + ports: + - port: 9090 + targetPort: 9090 + nodePort: 30901 + diff --git a/docs/examples/monitoring/builtin-prometheus/rbac/demo-2.yaml b/docs/examples/monitoring/builtin-prometheus/rbac/demo-2.yaml new file mode 100644 index 000000000..e6f011dd1 --- /dev/null +++ b/docs/examples/monitoring/builtin-prometheus/rbac/demo-2.yaml @@ -0,0 +1,90 @@ +apiVersion: rbac.authorization.k8s.io/v1beta1 +kind: ClusterRole +metadata: + name: prometheus-server +rules: +- apiGroups: [""] + resources: + - nodes + - nodes/proxy + - services + - endpoints + - pods + verbs: ["get", "list", "watch"] +- apiGroups: + - extensions + resources: + - ingresses + verbs: ["get", "list", "watch"] +- nonResourceURLs: ["/metrics"] + verbs: ["get"] +--- +apiVersion: v1 +kind: ServiceAccount +metadata: + name: prometheus-server + namespace: demo +--- +apiVersion: rbac.authorization.k8s.io/v1beta1 +kind: ClusterRoleBinding +metadata: + name: prometheus-server +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: prometheus-server +subjects: +- kind: ServiceAccount + name: prometheus-server + namespace: demo +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: prometheus-server + namespace: demo +spec: + replicas: 1 + selector: + matchLabels: + app: prometheus-server + template: + metadata: + labels: + app: prometheus-server + spec: + serviceAccountName: prometheus-server + containers: + - name: prometheus + image: prom/prometheus:v2.1.0 + args: + - "--config.file=/etc/prometheus/prometheus.yml" + - "--storage.tsdb.path=/prometheus/" + ports: + - containerPort: 9090 + volumeMounts: + - name: prometheus-config-volume + mountPath: /etc/prometheus/ + - name: prometheus-storage-volume + mountPath: /prometheus/ + volumes: + - name: prometheus-config-volume + configMap: + defaultMode: 420 + name: prometheus-server-conf + - name: prometheus-storage-volume + emptyDir: {} +--- +apiVersion: v1 +kind: Service +metadata: + name: prometheus-service + namespace: demo +spec: + selector: + app: prometheus-server + type: NodePort + ports: + - port: 9090 + targetPort: 9090 + nodePort: 30901 diff --git a/docs/examples/monitoring/demo-0.yaml b/docs/examples/monitoring/coreos-operator/demo-0.yaml similarity index 90% rename from docs/examples/monitoring/demo-0.yaml rename to docs/examples/monitoring/coreos-operator/demo-0.yaml index ce6ef4c8b..5e3348fe2 100644 --- a/docs/examples/monitoring/demo-0.yaml +++ b/docs/examples/monitoring/coreos-operator/demo-0.yaml @@ -22,7 +22,7 @@ spec: spec: containers: - name: prometheus-operator - image: quay.io/coreos/prometheus-operator:v0.10.1 + image: quay.io/coreos/prometheus-operator:v0.16.0 resources: requests: cpu: 100m diff --git a/docs/examples/monitoring/demo-1.yaml b/docs/examples/monitoring/coreos-operator/demo-1.yaml similarity index 91% rename from docs/examples/monitoring/demo-1.yaml rename to docs/examples/monitoring/coreos-operator/demo-1.yaml index f43ea6501..54221bb4f 100644 --- a/docs/examples/monitoring/demo-1.yaml +++ b/docs/examples/monitoring/coreos-operator/demo-1.yaml @@ -1,4 +1,4 @@ -apiVersion: monitoring.coreos.com/v1alpha1 +apiVersion: monitoring.coreos.com/v1 kind: Prometheus metadata: name: prometheus @@ -11,6 +11,7 @@ spec: resources: requests: memory: 400Mi + --- apiVersion: v1 kind: Service diff --git a/docs/examples/monitoring/coreos-operator/rbac/demo-0.yaml b/docs/examples/monitoring/coreos-operator/rbac/demo-0.yaml new file mode 100644 index 000000000..af0f7652d --- /dev/null +++ b/docs/examples/monitoring/coreos-operator/rbac/demo-0.yaml @@ -0,0 +1,105 @@ +apiVersion: v1 +kind: Namespace +metadata: + name: demo +spec: + finalizers: + - kubernetes +--- +apiVersion: rbac.authorization.k8s.io/v1beta1 +kind: ClusterRole +metadata: + name: prometheus-operator +rules: +- apiGroups: + - extensions + resources: + - thirdpartyresources + verbs: + - "*" +- apiGroups: + - apiextensions.k8s.io + resources: + - customresourcedefinitions + verbs: + - "*" +- apiGroups: + - monitoring.coreos.com + resources: + - alertmanagers + - prometheuses + - servicemonitors + verbs: + - "*" +- apiGroups: + - apps + resources: + - statefulsets + verbs: ["*"] +- apiGroups: [""] + resources: + - configmaps + - secrets + verbs: ["*"] +- apiGroups: [""] + resources: + - pods + verbs: ["list", "delete"] +- apiGroups: [""] + resources: + - services + - endpoints + verbs: ["get", "create", "update"] +- apiGroups: [""] + resources: + - nodes + verbs: ["list", "watch"] +- apiGroups: [""] + resources: + - namespaces + verbs: ["list"] +--- +apiVersion: v1 +kind: ServiceAccount +metadata: + name: prometheus-operator + namespace: demo +--- +apiVersion: rbac.authorization.k8s.io/v1beta1 +kind: ClusterRoleBinding +metadata: + name: prometheus-operator +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: prometheus-operator +subjects: +- kind: ServiceAccount + name: prometheus-operator + namespace: demo +--- +apiVersion: extensions/v1beta1 +kind: Deployment +metadata: + name: prometheus-operator + namespace: demo + labels: + operator: prometheus +spec: + replicas: 1 + template: + metadata: + labels: + operator: prometheus + spec: + serviceAccountName: prometheus-operator + containers: + - name: prometheus-operator + image: quay.io/coreos/prometheus-operator:v0.16.0 + resources: + requests: + cpu: 100m + memory: 50Mi + limits: + cpu: 200m + memory: 100Mi diff --git a/docs/examples/monitoring/coreos-operator/rbac/demo-1.yaml b/docs/examples/monitoring/coreos-operator/rbac/demo-1.yaml new file mode 100644 index 000000000..595da2f62 --- /dev/null +++ b/docs/examples/monitoring/coreos-operator/rbac/demo-1.yaml @@ -0,0 +1,72 @@ +apiVersion: rbac.authorization.k8s.io/v1beta1 +kind: ClusterRole +metadata: + name: prometheus +rules: +- apiGroups: [""] + resources: + - nodes + - services + - endpoints + - pods + verbs: ["get", "list", "watch"] +- apiGroups: [""] + resources: + - configmaps + verbs: ["get"] +- nonResourceURLs: ["/metrics"] + verbs: ["get"] + +--- +apiVersion: v1 +kind: ServiceAccount +metadata: + name: prometheus + namespace: demo + +--- +apiVersion: rbac.authorization.k8s.io/v1beta1 +kind: ClusterRoleBinding +metadata: + name: prometheus +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: prometheus +subjects: +- kind: ServiceAccount + name: prometheus + namespace: demo + +--- +apiVersion: monitoring.coreos.com/v1 +kind: Prometheus +metadata: + name: prometheus + namespace: demo +spec: + serviceAccountName: prometheus + serviceMonitorSelector: + matchLabels: + app: kubedb + version: v1.7.0 + resources: + requests: + memory: 400Mi + +--- +apiVersion: v1 +kind: Service +metadata: + name: prometheus + namespace: demo +spec: + type: LoadBalancer + ports: + - name: web + nodePort: 30900 + port: 9090 + protocol: TCP + targetPort: web + selector: + prometheus: prometheus diff --git a/docs/examples/mysql/demo-4.yaml b/docs/examples/mysql/demo-4.yaml index 1f53ce57a..921fa162c 100644 --- a/docs/examples/mysql/demo-4.yaml +++ b/docs/examples/mysql/demo-4.yaml @@ -15,3 +15,4 @@ spec: init: snapshotSource: name: m1-xyz + namespace: demo diff --git a/docs/guides/README.md b/docs/guides/README.md index b8303a782..83e498523 100644 --- a/docs/guides/README.md +++ b/docs/guides/README.md @@ -23,7 +23,7 @@ Guides show you how to perform tasks with KubeDB. - [Running MySQL](/docs/guides/mysql/overview.md): This tutorial will show you how to use KubeDB to run a MySQL database. - - [Running MongoDB](/docs/guides/mongodb/overview.md): This tutorial will show you how to use KubeDB to run a MongoDB database. + - [Running MongoDB](/docs/guides/mongodb/README.md): This tutorial will show you how to use KubeDB to run a MongoDB database. - [Running Redis](/docs/guides/redis/overview.md): This tutorial will show you how to use KubeDB to run a Redis database. diff --git a/docs/guides/mongodb/README.md b/docs/guides/mongodb/README.md new file mode 100644 index 000000000..1654efb0b --- /dev/null +++ b/docs/guides/mongodb/README.md @@ -0,0 +1,46 @@ +> New to KubeDB? Please start [here](/docs/guides/README.md). + +## MongoDB versions supported by KubeDB + +| KubeDB Version | Mongo:3.4 | Mongo:3.6 | +|:--:|:--:|:--:| +| 0.1.0 - 0.7.0 | ✗ | ✗ | +| 0.8.0-beta.0 | ✓ | ✓ | +| 0.8.0-beta.1 | ✓ | ✓ | + +
+ +## KubeDB Features and their availability for MongoDB + +|Features |Availability| +|--|:--:| +|Clustering | ✗ | +|Persistent Volume | ✓ | +|Instant Backup | ✓ | +|Scheduled Backup | ✓ | +|Initialize using Snapshot | ✓ | +|Initialize using Script (\*.js and/or \*.sh) | ✓ | +|out-of-the-box builtin-Prometheus Monitoring | ✓ | +|out-of-the-box CoreOS-Prometheus-Operator Monitoring | ✓ | + +
+ +## Life Cycle of MongoDB in KubeDB +

+  lifecycle +

+ + +## User Guide +- [Quickstart MongoDB](/docs/guides/mongodb/quickstart/quickstart.md) with KubeDB Operator. +- [Snapshot and Restore](/docs/guides/mongodb/snapshot/backup-and-restore.md) process of MongoDB databases using KubeDB. +- Take [Scheduled Snapshot](/docs/guides/mongodb/snapshot/scheduled-backup.md) of MongoDB databases using KubeDB. +- Initialize [MongoDB with Script](/docs/guides/mongodb/initialization/using-script.md). +- Initialize [MongoDB with Snapshot](/docs/guides/mongodb/initialization/using-snapshot.md). +- Monitor your MongoDB database with KubeDB using [out-of-the-box CoreOS Prometheus Operator](/docs/guides/mongodb/monitoring/using-coreos-prometheus-operator.md). +- Monitor your MongoDB database with KubeDB using [out-of-the-box builtin-Prometheus](/docs/guides/mongodb/monitoring/using-builtin-prometheus.md). +- Use [Private Docker Registry](/docs/guides/mongodb/private-registry/using-private-registry.md) to deploy MongoDB with KubeDB. +- Detail concepts of [MongoDB object](/docs/concepts/databases/mongodb.md). +- Detail concepts of [Snapshot object](/docs/concepts/snapshot.md). +- Wondering what features are coming next? Please visit [here](/docs/roadmap.md). +- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/mongodb/initialization/using-script.md b/docs/guides/mongodb/initialization/using-script.md new file mode 100644 index 000000000..914b79aea --- /dev/null +++ b/docs/guides/mongodb/initialization/using-script.md @@ -0,0 +1,268 @@ +> New to KubeDB? Please start [here](/docs/guides/README.md). + +# Initialize MongoDB using Script + +This tutorial will show you how to use KubeDB to initialize a MongoDB database with .js and/or .sh script. + +## Before You Begin +At first, you need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one by using [Minikube](https://github.com/kubernetes/minikube). + +Now, install KubeDB cli on your workstation and KubeDB operator in your cluster following the steps [here](/docs/setup/install.md). + +To keep things isolated, this tutorial uses a separate namespace called `demo` throughout this tutorial. Run the following command to prepare your cluster for this tutorial: + +```console +$ kubectl create -f https://raw.githubusercontent.com/kubedb/cli/0.8.0-beta.1/docs/examples/mongodb/demo-0.yaml +namespace "demo" created + +$ kubectl get ns +NAME STATUS AGE +default Active 45m +demo Active 10s +kube-public Active 45m +kube-system Active 45m +``` + +In this tutorial we will use .js script stored in GitHub repository [kubedb/mongodb-init-scripts](https://github.com/kubedb/mongodb-init-scripts). + +Please note that the yaml files that are used in this tutorial, stored in [docs/examples](https://github.com/kubedb/cli/tree/master/docs/examples) folder in GitHub repository [kubedb/cli](https://github.com/kubedb/cli). + +## Create a MongoDB database with Init-Script +Below is the `MongoDB` object created in this tutorial. + +```yaml +apiVersion: kubedb.com/v1alpha1 +kind: MongoDB +metadata: + name: mgo-init-script + namespace: demo +spec: + version: 3.4 + doNotPause: true + storage: + storageClassName: "standard" + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 50Mi + init: + scriptSource: + gitRepo: + repository: "https://github.com/kubedb/mongodb-init-scripts.git" + directory: . + +``` + +```console +$ kubedb create -f https://raw.githubusercontent.com/kubedb/cli/0.8.0-beta.1/docs/examples/mongodb/Initialization/demo-1.yaml +validating "https://raw.githubusercontent.com/kubedb/cli/0.8.0-beta.1/docs/examples/mongodb/Initialization/demo-1.yaml" +mongodb "mgo-init-script" created +``` + + +Here, + + - `spec.version` is the version of MongoDB database. In this tutorial, a MongoDB 3.4 database is going to be created. + + - `spec.storage` specifies the StorageClass of PVC dynamically allocated to store data for this database. This storage spec will be passed to the StatefulSet created by KubeDB operator to run database pods. You can specify any StorageClass available in your cluster with appropriate resource requests. If no storage spec is given, an `emptyDir` is used. + + - `spec.init.scriptSource` specifies a script source used to initialize the database before database server starts. The scripts will be executed alphabatically. In this tutorial, a sample .js script from the git repository `https://github.com/kubedb/mongodb-init-scripts.git` is used to create a test database. You can use other [volume sources](https://kubernetes.io/docs/concepts/storage/volumes/#types-of-volumes) instead of `gitrepo`. The \*.js and/or \*.sh sripts that are stored inside the root folder will be executed alphabatically. The scripts inside child folders will be skipped. + +KubeDB operator watches for `MongoDB` objects using Kubernetes api. When a `MongoDB` object is created, KubeDB operator will create a new StatefulSet and a ClusterIP Service with the matching MongoDB object name. KubeDB operator will also create a governing service for StatefulSets with the name `kubedb`, if one is not already present. No MongoDB specific RBAC roles are required for [RBAC enabled clusters](/docs/guides/rbac.md). + +```console +$ kubedb describe mg -n demo mgo-init-script +Name: mgo-init-script +Namespace: demo +StartTimestamp: Tue, 06 Feb 2018 09:56:07 +0600 +Status: Running +Volume: + StorageClass: standard + Capacity: 50Mi + Access Modes: RWO + +StatefulSet: + Name: mgo-init-script + Replicas: 1 current / 1 desired + CreationTimestamp: Tue, 06 Feb 2018 09:56:12 +0600 + Pods Status: 1 Running / 0 Waiting / 0 Succeeded / 0 Failed + +Service: + Name: mgo-init-script + Type: ClusterIP + IP: 10.106.175.209 + Port: db 27017/TCP + +Database Secret: + Name: mgo-init-script-auth + Type: Opaque + Data + ==== + password: 16 bytes + user: 4 bytes + +No Snapshots. + +Events: + FirstSeen LastSeen Count From Type Reason Message + --------- -------- ----- ---- -------- ------ ------- + 6s 6s 1 MongoDB operator Normal Successful Successfully patched StatefulSet + 6s 6s 1 MongoDB operator Normal Successful Successfully patched MongoDB + 9s 9s 1 MongoDB operator Normal Successful Successfully created StatefulSet + 9s 9s 1 MongoDB operator Normal Successful Successfully created MongoDB + 18s 18s 1 MongoDB operator Normal Successful Successfully created Service + + + +$ kubectl get statefulset -n demo +NAME DESIRED CURRENT AGE +mgo-init-script 1 1 46s + + + +$ kubectl get pvc -n demo +NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE +data-mgo-init-script-0 Bound pvc-ac84fbb9-0af1-11e8-a107-080027869227 50Mi RWO standard 1m + + + +$ kubectl get pv -n demo +NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE +pvc-ac84fbb9-0af1-11e8-a107-080027869227 50Mi RWO Delete Bound demo/data-mgo-init-script-0 standard + + +$ kubectl get service -n demo +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +kubedb ClusterIP None 2m +mgo-init-script ClusterIP 10.106.175.209 27017/TCP 2m +``` + + +KubeDB operator sets the `status.phase` to `Running` once the database is successfully created. Run the following command to see the modified MongoDB object: + +```yaml +$ kubedb get mg -n demo mgo-init-script -o yaml +apiVersion: kubedb.com/v1alpha1 +kind: MongoDB +metadata: + clusterName: "" + creationTimestamp: 2018-02-06T03:56:07Z + finalizers: + - kubedb.com + generation: 0 + name: mgo-init-script + namespace: demo + resourceVersion: "4827" + selfLink: /apis/kubedb.com/v1alpha1/namespaces/demo/mongodbs/mgo-init-script + uid: a9348cad-0af1-11e8-a107-080027869227 +spec: + databaseSecret: + secretName: mgo-init-script-auth + doNotPause: true + init: + scriptSource: + gitRepo: + directory: . + repository: https://github.com/kubedb/mongodb-init-scripts.git + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 50Mi + storageClassName: standard + version: 3.4 +status: + creationTime: 2018-02-06T03:56:12Z + phase: Running +``` + +Please note that KubeDB operator has created a new Secret called `mgo-init-script-auth` *(format: {mongodb-object-name}-auth)* for storing the password for MongoDB superuser. This secret contains a `user` key which contains the *username* for MongoDB superuser and a `password` key which contains the *password* for MongoDB superuser. +If you want to use an existing secret please specify that when creating the MongoDB object using `spec.databaseSecret.secretName`. While creating this secret manually, make sure the secret contains these two keys containing data `user` and `password`. + +```console +$ kubectl get secrets -n demo mgo-init-script-auth -o json +apiVersion: v1 +data: + password: STJ1YnNiU3BUNzFOZUhXSA== + user: cm9vdA== +kind: Secret +metadata: + creationTimestamp: 2018-02-06T03:56:12Z + labels: + kubedb.com/kind: MongoDB + kubedb.com/name: mgo-init-script + name: mgo-init-script-auth + namespace: demo + resourceVersion: "4789" + selfLink: /api/v1/namespaces/demo/secrets/mgo-init-script-auth + uid: ac33c72d-0af1-11e8-a107-080027869227 +type: Opaque +``` + + +Now, you can connect to this database through [mongo-shell](https://docs.mongodb.com/v3.4/mongo/). In this tutorial, we are connecting to the MongoDB server from inside the pod. + +```console +$ kubectl get secrets -n demo mgo-init-script-auth -o jsonpath='{.data.\user}' | base64 -d +root + +$ kubectl get secrets -n demo mgo-init-script-auth -o jsonpath='{.data.\password}' | base64 -d +I2ubsbSpT71NeHWH + +$ kubectl exec -it mgo-init-script-0 -n demo sh + +> mongo admin +MongoDB shell version v3.4.10 +connecting to: mongodb://127.0.0.1:27017/admin +MongoDB server version: 3.4.10 +Welcome to the MongoDB shell. +For interactive help, type "help". +For more comprehensive documentation, see + http://docs.mongodb.org/ +Questions? Try the support group + http://groups.google.com/group/mongodb-user + +> db.auth("root","I2ubsbSpT71NeHWH") +1 + +> show dbs +admin 0.000GB +local 0.000GB +mydb 0.000GB + +> use mydb +switched to db mydb + +> db.movie.find() +{ "_id" : ObjectId("5a72b2b1e1a0770e3bdb56f1"), "name" : "batman" } + +> exit +bye +``` + +As you can see here, the initial script has successfully created a database named `mydb` and inserted data into that database successfully. + +## Cleaning up +To cleanup the Kubernetes resources created by this tutorial, run: + +```console +$ kubedb delete mg,drmn,snap -n demo --all --force + +$ kubectl delete ns demo +namespace "demo" deleted +``` + + +## Next Steps +- Initialize [MongoDB with Snapshot](/docs/guides/mongodb/initialization/using-snapshot.md). +- [Snapshot and Restore](/docs/guides/mongodb/snapshot/backup-and-restore.md) process of MongoDB databases using KubeDB. +- Take [Scheduled Snapshot](/docs/guides/mongodb/snapshot/scheduled-backup.md) of MongoDB databases using KubeDB. +- Initialize [MongoDB with Script](/docs/guides/mongodb/initialization/using-script.md). +- Monitor your MongoDB database with KubeDB using [out-of-the-box CoreOS Prometheus Operator](/docs/guides/mongodb/monitoring/using-coreos-prometheus-operator.md). +- Monitor your MongoDB database with KubeDB using [out-of-the-box builtin-Prometheus](/docs/guides/mongodb/monitoring/using-builtin-prometheus.md). +- Use [Private Docker Registry](/docs/guides/mongodb/private-registry/using-private-registry.md) to deploy MongoDB with KubeDB. +- Detail concepts of [MongoDB object](/docs/concepts/databases/mongodb.md). +- Wondering what features are coming next? Please visit [here](/docs/roadmap.md). +- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/mongodb/initialization/using-snapshot.md b/docs/guides/mongodb/initialization/using-snapshot.md new file mode 100644 index 000000000..b4b217430 --- /dev/null +++ b/docs/guides/mongodb/initialization/using-snapshot.md @@ -0,0 +1,123 @@ +> New to KubeDB? Please start [here](/docs/guides/README.md). + +# Initialize MongoDB with Snapshot +This tutorial will show you how to use KubeDB to initialize a MongoDB database with an existing snapshot. + +## Before You Begin +At first, you need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one by using [Minikube](https://github.com/kubernetes/minikube). + +Now, install KubeDB cli on your workstation and KubeDB operator in your cluster following the steps [here](/docs/setup/install.md). + +This tutorial assumes that you have created a namespace `demo` and a snapshot `snapshot-infant`. Follow the steps [here](/docs/guides/mongodb/snapshot/backup-and-restore.md) to create a database and take [instant snapshot](/docs/guides/mongodb/snapshot/backup-and-restore.md#instant-backups), if you have not done so already. If you have changed the name of either namespace or snapshot object, please modify the YAMLs used in this tutorial accordingly. + +Note that the yaml files that are used in this tutorial, stored in [docs/examples](https://github.com/kubedb/cli/tree/master/docs/examples) folder in GitHub repository [kubedb/cli](https://github.com/kubedb/cli). + +## Create MongoDB with Init-Snapshot +Below is the `MongoDB` object created in this tutorial. + +```yaml +apiVersion: kubedb.com/v1alpha1 +kind: MongoDB +metadata: + name: mgo-init-snapshot + namespace: demo +spec: + version: 3.4 + storage: + storageClassName: "standard" + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 50Mi + init: + snapshotSource: + name: snapshot-infant + namespace: demo +``` + +```console +$ kubedb create -f https://raw.githubusercontent.com/kubedb/cli/0.8.0-beta.1/docs/examples/mongodb/Initialization/demo-2.yaml +validating "https://raw.githubusercontent.com/kubedb/cli/0.8.0-beta.1/docs/examples/mongodb/Initialization/demo-2.yaml" +mongodb "mgo-init-snapshot" created +``` + +Here, + +- `spec.init.snapshotSource.name` refers to a Snapshot object for a MongoDB database in the same namespaces as this new `mgo-init-snapshot` MongoDB object. + +Now, wait several seconds. KubeDB operator will create a new `StatefulSet`. Then KubeDB operator launches a Kubernetes Job to initialize the new database using the data from `snapshot-infant` Snapshot. + +```console +$ kubedb get mg -n demo +NAME STATUS AGE +mgo-infant Running 24m +mgo-init-snapshot Initializing 6s + + +$ kubedb describe mg -n demo mgo-init-snapshot +Name: mgo-init-snapshot +Namespace: demo +StartTimestamp: Tue, 06 Feb 2018 10:34:30 +0600 +Status: Running +Annotations: kubedb.com/initialized= +Volume: + StorageClass: standard + Capacity: 50Mi + Access Modes: RWO + +StatefulSet: + Name: mgo-init-snapshot + Replicas: 1 current / 1 desired + CreationTimestamp: Tue, 06 Feb 2018 10:11:54 +0600 + Pods Status: 1 Running / 0 Waiting / 0 Succeeded / 0 Failed + +Service: + Name: mgo-init-snapshot + Type: ClusterIP + IP: 10.100.233.80 + Port: db 27017/TCP + +Database Secret: + Name: mgo-init-snapshot-auth + Type: Opaque + Data + ==== + password: 16 bytes + user: 4 bytes + +No Snapshots. + +Events: + FirstSeen LastSeen Count From Type Reason Message + --------- -------- ----- ---- -------- ------ ------- + 1m 1m 1 MongoDB operator Normal Successful Successfully patched StatefulSet + 1m 1m 1 MongoDB operator Normal Successful Successfully patched MongoDB + 1m 1m 1 Job Controller Normal SuccessfulInitialize Successfully completed initialization + 1m 1m 1 MongoDB operator Normal Successful Successfully patched StatefulSet + 1m 1m 1 MongoDB operator Normal Successful Successfully patched MongoDB + 1m 1m 1 MongoDB operator Normal Initializing Initializing from Snapshot: "snapshot-infant" + 1m 1m 1 MongoDB operator Normal Successful Successfully patched StatefulSet + 1m 1m 1 MongoDB operator Normal Successful Successfully patched MongoDB +``` + + +## Cleaning up +To cleanup the Kubernetes resources created by this tutorial, run: + +```console +$ kubedb delete mg,drmn,snap -n demo --all --force + +$ kubectl delete ns demo +namespace "demo" deleted +``` + + +## Next Steps +- Monitor your MongoDB database with KubeDB using [out-of-the-box CoreOS Prometheus Operator](/docs/guides/mongodb/monitoring/using-coreos-prometheus-operator.md). +- Monitor your MongoDB database with KubeDB using [out-of-the-box builtin-Prometheus](/docs/guides/mongodb/monitoring/using-builtin-prometheus.md). +- Use [Private Docker Registry](/docs/guides/mongodb/private-registry/using-private-registry.md) to deploy MongoDB with KubeDB. +- Detail concepts of [MongoDB object](/docs/concepts/databases/mongodb.md). +- Detail concepts of [Snapshot object](/docs/concepts/snapshot.md). +- Wondering what features are coming next? Please visit [here](/docs/roadmap.md). +- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/mongodb/monitoring/using-builtin-prometheus.md b/docs/guides/mongodb/monitoring/using-builtin-prometheus.md new file mode 100644 index 000000000..ec83297f6 --- /dev/null +++ b/docs/guides/mongodb/monitoring/using-builtin-prometheus.md @@ -0,0 +1,368 @@ +> New to KubeDB? Please start [here](/docs/guides/README.md). + +# Using Prometheus with KubeDB +This tutorial will show you how to monitor KubeDB databases using [Prometheus](https://prometheus.io/). + +## Before You Begin +At first, you need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one by using [Minikube](https://github.com/kubernetes/minikube). + +Now, install KubeDB cli on your workstation and KubeDB operator in your cluster following the steps [here](/docs/setup/install.md). + +To keep things isolated, this tutorial uses a separate namespace called `demo` throughout this tutorial. Run the following command to prepare your cluster for this tutorial: + +```console +$ kubectl create -f https://raw.githubusercontent.com/kubedb/cli/0.8.0-beta.1/docs/examples/mongodb/demo-0.yaml +namespace "demo" created + +$ kubectl get ns +NAME STATUS AGE +default Active 45m +demo Active 10s +kube-public Active 45m +kube-system Active 45m +``` + +Please note that the yaml files that are used in this tutorial, stored in [docs/examples](https://github.com/kubedb/cli/tree/master/docs/examples) folder in GitHub repository [kubedb/cli](https://github.com/kubedb/cli). + +## Create a MongoDB database +KubeDB implements a `MongoDB` CRD to define the specification of a MongoDB database. Below is the `MongoDB` object created in this tutorial. + +```yaml +apiVersion: kubedb.com/v1alpha1 +kind: MongoDB +metadata: + name: mgo-mon-prometheus + namespace: demo +spec: + version: 3.4 + storage: + storageClassName: "standard" + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 50Mi + monitor: + agent: prometheus.io/builtin +``` + +```console +$ kubedb create -f https://raw.githubusercontent.com/kubedb/cli/0.8.0-beta.1/docs/examples/mongodb/monitoring/builtin-prometheus/demo-1.yaml +validating "https://raw.githubusercontent.com/kubedb/cli/0.8.0-beta.1/docs/examples/mongodb/monitoring/builtin-prometheus/demo-1.yaml" +mongodb "mgo-mon-prometheus" created +``` + + +Here, + + - `spec.version` is the version of MongoDB database. In this tutorial, a MongoDB 3.4 database is going to be created. + + - `spec.storage` specifies the StorageClass of PVC dynamically allocated to store data for this database. This storage spec will be passed to the StatefulSet created by KubeDB operator to run database pods. You can specify any StorageClass available in your cluster with appropriate resource requests. If no storage spec is given, an `emptyDir` is used. + + - `spec.monitor` specifies that built-in [Prometheus](https://github.com/prometheus/prometheus) is used to monitor this database instance. KubeDB operator will configure the service of this database in a way that the Prometheus server will automatically find out the service endpoint aka `MongoDB Exporter` and will receive metrics from exporter. + +KubeDB operator watches for `MongoDB` objects using Kubernetes api. When a `MongoDB` object is created, KubeDB operator will create a new StatefulSet and a ClusterIP Service with the matching crd name. KubeDB operator will also create a governing service for StatefulSets with the name `kubedb`, if one is not already present. + +```console +$ kubedb get mg -n demo +NAME STATUS AGE +mgo-mon-prometheus Creating 30s + + +$ kubedb get mg -n demo +NAME STATUS AGE +mgo-mon-prometheus Running 10m + +$ kubedb describe mg -n demo mgo-mon-prometheus +Name: mgo-mon-prometheus +Namespace: demo +StartTimestamp: Mon, 05 Feb 2018 17:38:29 +0600 +Status: Running +Volume: + StorageClass: standard + Capacity: 50Mi + Access Modes: RWO + +StatefulSet: + Name: mgo-mon-prometheus + Replicas: 1 current / 1 desired + CreationTimestamp: Mon, 05 Feb 2018 17:37:54 +0600 + Pods Status: 1 Running / 0 Waiting / 0 Succeeded / 0 Failed + +Service: + Name: mgo-mon-prometheus + Type: ClusterIP + IP: 10.104.88.103 + Port: db 27017/TCP + Port: prom-http 56790/TCP + +Database Secret: + Name: mgo-mon-prometheus-auth + Type: Opaque + Data + ==== + user: 4 bytes + password: 16 bytes + +Monitoring System: + Agent: prometheus.io/builtin + Prometheus: + Namespace: + Interval: + +No Snapshots. + +Events: + FirstSeen LastSeen Count From Type Reason Message + --------- -------- ----- ---- -------- ------ ------- + 15m 15m 1 MongoDB operator Normal Successful Successfully patched StatefulSet + 15m 15m 1 MongoDB operator Normal Successful Successfully patched MongoDB + 15m 15m 1 MongoDB operator Normal Successful Successfully patched StatefulSet + 15m 15m 1 MongoDB operator Normal Successful Successfully patched MongoDB +``` + + +Since `spec.monitoring` was configured, the database service object is configured accordingly. You can verify it running the following commands: + +```yaml +$ kubectl get services -n demo +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +kubedb ClusterIP None 22m +mgo-mon-prometheus ClusterIP 10.104.88.103 27017/TCP,56790/TCP 22m +``` + +```yaml +$ kubectl get services mgo-mon-prometheus -n demo -o yaml +apiVersion: v1 +kind: Service +metadata: + annotations: + monitoring.appscode.com/agent: prometheus.io/builtin + prometheus.io/path: /kubedb.com/v1alpha1/namespaces/demo/mongodbs/mgo-mon-prometheus/metrics + prometheus.io/port: "56790" + prometheus.io/scrape: "true" + creationTimestamp: 2018-02-05T11:37:53Z + labels: + kubedb.com/kind: MongoDB + kubedb.com/name: mgo-mon-prometheus + name: mgo-mon-prometheus + namespace: demo + resourceVersion: "1709" + selfLink: /api/v1/namespaces/demo/services/mgo-mon-prometheus + uid: 00a3de77-0a69-11e8-9639-080027869227 +spec: + clusterIP: 10.104.88.103 + ports: + - name: db + port: 27017 + protocol: TCP + targetPort: db + - name: prom-http + port: 56790 + protocol: TCP + targetPort: prom-http + selector: + kubedb.com/kind: MongoDB + kubedb.com/name: mgo-mon-prometheus + sessionAffinity: None + type: ClusterIP +status: + loadBalancer: {} +``` + +We can see that the service contains these specific annotations. The Prometheus server will discover the exporter using these specifications. + +```yaml +prometheus.io/path: ... +prometheus.io/port: ... +prometheus.io/scrape: ... +``` + +## Deploy and configure Prometheus Server + +The Prometheus server is needed to configure so that it can discover endpoints of services. If a Prometheus server is already running in cluster and if it is configured in a way that it can discover service endpoints, no extra configuration will be needed. If there is no existing Prometheus server running, rest of this tutorial will create a Prometheus server with appropriate configuration. + +The configuration file to `Prometheus-Server` will be provided by `ConfigMap`. The below config map will be created: + +```yaml +apiVersion: v1 +kind: ConfigMap +metadata: + name: prometheus-server-conf + labels: + name: prometheus-server-conf + namespace: demo +data: + prometheus.yml: |- + global: + scrape_interval: 5s + evaluation_interval: 5s + scrape_configs: + - job_name: 'kubernetes-service-endpoints' + + kubernetes_sd_configs: + - role: endpoints + + relabel_configs: + - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scrape] + action: keep + regex: true + - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scheme] + action: replace + target_label: __scheme__ + regex: (https?) + - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_path] + action: replace + target_label: __metrics_path__ + regex: (.+) + - source_labels: [__address__, __meta_kubernetes_service_annotation_prometheus_io_port] + action: replace + target_label: __address__ + regex: ([^:]+)(?::\d+)?;(\d+) + replacement: $1:$2 + - action: labelmap + regex: __meta_kubernetes_service_label_(.+) + - source_labels: [__meta_kubernetes_namespace] + action: replace + target_label: kubernetes_namespace + - source_labels: [__meta_kubernetes_service_name] + action: replace + target_label: kubernetes_name +``` + +```console +$ kubectl create -f https://raw.githubusercontent.com/kubedb/cli/0.8.0-beta.1/docs/examples/monitoring/builtin-prometheus/demo-1.yaml +configmap "prometheus-server-conf" created +``` + +Now, the below yaml is used to deploy Prometheus in kubernetes : + +```yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: prometheus-server + namespace: demo +spec: + replicas: 1 + selector: + matchLabels: + app: prometheus-server + template: + metadata: + labels: + app: prometheus-server + spec: + containers: + - name: prometheus + image: prom/prometheus:v2.1.0 + args: + - "--config.file=/etc/prometheus/prometheus.yml" + - "--storage.tsdb.path=/prometheus/" + ports: + - containerPort: 9090 + volumeMounts: + - name: prometheus-config-volume + mountPath: /etc/prometheus/ + - name: prometheus-storage-volume + mountPath: /prometheus/ + volumes: + - name: prometheus-config-volume + configMap: + defaultMode: 420 + name: prometheus-server-conf + - name: prometheus-storage-volume + emptyDir: {} +``` + +#### In RBAC enabled cluster +If RBAC *is* enabled, Run the following command to deploy prometheus in kubernetes: + +```console +$ kubectl create -f https://raw.githubusercontent.com/kubedb/cli/0.8.0-beta.1/docs/examples/monitoring/builtin-prometheus/rbac/demo-2.yaml +clusterrole "prometheus-server" created +serviceaccount "prometheus-server" created +clusterrolebinding "prometheus-server" created +deployment "prometheus-server" created +service "prometheus-service" created + +# Verify RBAC stuffs +$ kubectl get clusterroles +NAME AGE +prometheus-server 57s + +$ kubectl get clusterrolebindings +NAME AGE +prometheus-server 1m + + +$ kubectl get serviceaccounts -n demo +NAME SECRETS AGE +default 1 48m +prometheus-server 1 1m +``` + + +#### In RBAC \*not\* enabled cluster +If RBAC *is not* enabled, Run the following command to prepare your cluster for this tutorial: + +```console +$ kubectl create -f https://raw.githubusercontent.com/kubedb/cli/0.8.0-beta.1/docs/examples/monitoring/builtin-prometheus/demo-2.yaml +deployment "prometheus-server" created +service "prometheus-service" created + + +$ kubectl get pods -n demo --watch +NAME READY STATUS RESTARTS AGE +mgo-mon-prometheus-0 2/2 Running 0 48m +prometheus-server-79c7cf44fc-m95lm 1/1 Running 0 34s +``` + +#### Prometheus Dashboard +Now to open prometheus dashboard on Browser: + +```console +$ kubectl get svc -n demo +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +kubedb ClusterIP None 59m +mgo-mon-prometheus ClusterIP 10.104.88.103 27017/TCP,56790/TCP 59m +prometheus-service LoadBalancer 10.103.201.246 9090:30901/TCP 8s + + +$ minikube ip +192.168.99.100 + +$ minikube service prometheus-service -n demo --url +http://192.168.99.100:30901 +``` + +Now, open your browser and go to the following URL: _http://{minikube-ip}:{prometheus-svc-nodeport}_ to visit Prometheus Dashboard. According to the above example, this URL will be [http://192.168.99.100:30901](http://192.168.99.100:30901). + +Now, if you go the Prometheus Dashboard, you should see that this database endpoint as one of the targets. +![prometheus-builtin](/docs/images/mongodb/builtin-prometheus.png) + +## Cleaning up +To cleanup the Kubernetes resources created by this tutorial, run: + +```console +$ kubedb delete mg,drmn,snap -n demo --all --force + +# In rbac enabled cluster, +# $ kubectl delete clusterrole prometheus-server +# $ kubectl delete clusterrolebindings prometheus-server +# $ kubectl delete serviceaccounts -n demo prometheus-server + +$ kubectl delete ns demo +namespace "demo" deleted +``` + +## Next Steps +- Monitor your MongoDB database with KubeDB using [out-of-the-box CoreOS Prometheus Operator](/docs/guides/mongodb/monitoring/using-coreos-prometheus-operator.md). +- Detail concepts of [MongoDB object](/docs/concepts/databases/mongodb.md). +- [Snapshot and Restore](/docs/guides/mongodb/snapshot/backup-and-restore.md) process of MongoDB databases using KubeDB. +- Take [Scheduled Snapshot](/docs/guides/mongodb/snapshot/scheduled-backup.md) of MongoDB databases using KubeDB. +- Initialize [MongoDB with Script](/docs/guides/mongodb/initialization/using-script.md). +- Initialize [MongoDB with Snapshot](/docs/guides/mongodb/initialization/using-snapshot.md). +- Use [Private Docker Registry](/docs/guides/mongodb/private-registry/using-private-registry.md) to deploy MongoDB with KubeDB. +- Wondering what features are coming next? Please visit [here](/docs/roadmap.md). +- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/mongodb/monitoring/using-coreos-prometheus-operator.md b/docs/guides/mongodb/monitoring/using-coreos-prometheus-operator.md new file mode 100644 index 000000000..85cb4e0e0 --- /dev/null +++ b/docs/guides/mongodb/monitoring/using-coreos-prometheus-operator.md @@ -0,0 +1,319 @@ + +> New to KubeDB? Please start [here](/docs/guides/README.md). + +# Using Prometheus (CoreOS operator) with KubeDB +This tutorial will show you how to monitor KubeDB databases using Prometheus via [CoreOS Prometheus Operator](https://github.com/coreos/prometheus-operator). + +## Before You Begin +At first, you need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one by using [Minikube](https://github.com/kubernetes/minikube). + +Now, install KubeDB cli on your workstation and KubeDB operator in your cluster following the steps [here](/docs/setup/install.md). + +To keep things isolated, this tutorial uses a separate namespace called `demo` throughout this tutorial. + +Please note that the yaml files that are used in this tutorial, stored in [docs/examples](https://github.com/kubedb/cli/tree/master/docs/examples) folder in GitHub repository [kubedb/cli](https://github.com/kubedb/cli). + +## Deploy CoreOS-Prometheus Operator + +#### In RBAC enabled cluster +If RBAC *is* enabled, Run the following command to prepare your cluster for this tutorial: + +```console +$ kubectl create -f https://raw.githubusercontent.com/kubedb/cli/0.8.0-beta.1/docs/examples/monitoring/coreos-operator/rbac/demo-0.yaml +namespace "demo" created +clusterrole "prometheus-operator" created +serviceaccount "prometheus-operator" created +clusterrolebinding "prometheus-operator" created +deployment "prometheus-operator" created + +$ kubectl get pods -n demo --watch +NAME READY STATUS RESTARTS AGE +prometheus-operator-79cb9dcd4b-2njgq 1/1 Running 0 2m + + +$ kubectl get crd +NAME AGE +alertmanagers.monitoring.coreos.com 11m +prometheuses.monitoring.coreos.com 11m +servicemonitors.monitoring.coreos.com 11m +``` + + +Once the Prometheus operator CRDs are registered, run the following command to create a Prometheus. + +```console +$ kubectl create -f https://raw.githubusercontent.com/kubedb/cli/0.8.0-beta.1/docs/examples/monitoring/coreos-operator/rbac/demo-1.yaml +clusterrole "prometheus" created +serviceaccount "prometheus" created +clusterrolebinding "prometheus" created +prometheus "prometheus" created +service "prometheus" created + +# Verify RBAC stuffs +$ kubectl get clusterroles +NAME AGE +prometheus 48s +prometheus-operator 1m + + +$ kubectl get clusterrolebindings +NAME AGE +prometheus 7s +prometheus-operator 25s + + +$ kubectl get serviceaccounts -n demo +NAME SECRETS AGE +default 1 5m +prometheus 1 4m +prometheus-operator 1 5m +``` + + +#### In RBAC \*not\* enabled cluster +If RBAC *is not* enabled, Run the following command to prepare your cluster for this tutorial: + +```console +$ kubectl create -f https://raw.githubusercontent.com/kubedb/cli/0.8.0-beta.1/docs/examples/monitoring/coreos-operator/demo-0.yaml +namespace "demo" created +deployment "prometheus-operator" created + + +$ kubectl get pods -n demo --watch +NAME READY STATUS RESTARTS AGE +prometheus-operator-5dcd844486-nprmk 0/1 ContainerCreating 0 27s +prometheus-operator-5dcd844486-nprmk 1/1 Running 0 46s + + +$ kubectl get crd +NAME AGE +alertmanagers.monitoring.coreos.com 45s +prometheuses.monitoring.coreos.com 44s +servicemonitors.monitoring.coreos.com 44s +``` + + +Once the Prometheus operator CRDs are registered, run the following command to create a Prometheus. + +```console +$ kubectl create -f https://raw.githubusercontent.com/kubedb/cli/0.8.0-beta.1/docs/examples/monitoring/coreos-operator/demo-1.yaml +prometheus "prometheus" created +service "prometheus" created + +``` + +#### Prometheus Dashboard +Now to open prometheus dashboard on Browser: + +```console +$ kubectl get svc -n demo +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +prometheus LoadBalancer 10.99.201.154 9090:30900/TCP 5m +prometheus-operated ClusterIP None 9090/TCP 5m + +$ minikube ip +192.168.99.100 + +$ minikube service prometheus -n demo --url +http://192.168.99.100:30900 +``` + + +Now, open your browser and go to the following URL: _http://{minikube-ip}:{prometheus-svc-nodeport}_ to visit Prometheus Dashboard. According to the above example, this URL will be [http://192.168.99.100:30900](http://192.168.99.100:30900). + +## Create a MongoDB database +KubeDB implements a `MongoDB` CRD to define the specification of a MongoDB database. Below is the `MongoDB` object created in this tutorial. + +```yaml +apiVersion: kubedb.com/v1alpha1 +kind: MongoDB +metadata: + name: mgo-mon-coreos + namespace: demo +spec: + version: 3.4 + storage: + storageClassName: "standard" + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 50Mi + monitor: + agent: prometheus.io/coreos-operator + prometheus: + namespace: demo + labels: + app: kubedb + interval: 10s +``` + +The `MongoDB` CRD object contains `monitor` field in it's `spec`. It is also possible to add CoreOS-Prometheus monitor to an existing `MongoDB` database by adding the below part in it's `spec` field. +```yaml +spec: + monitor: + agent: prometheus.io/coreos-operator + prometheus: + namespace: demo + labels: + app: kubedb + interval: 10s +``` + +| Keys | Value | Description | +|--|--|--| +| `spec.monitor.agent` | string | `Required`. Indicates the monitoring agent used. Only valid value currently is `coreos-prometheus-operator` | +| `spec.monitor.prometheus.namespace` | string | `Required`. Indicates namespace where service monitors are created. This must be the same namespace of the Prometheus instance. | +| `spec.monitor.prometheus.labels` | map | `Required`. Indicates labels applied to service monitor. | +| `spec.monitor.prometheus.interval` | string | `Optional`. Indicates the scrape interval for database exporter endpoint (eg, '10s') | +| `spec.monitor.prometheus.port` | int |`Optional`. Indicates the port for database exporter endpoint (default is `56790`)| + +__Known Limitations:__ If the database password is updated, exporter must be restarted to use the new credentials. This issue is tracked [here](https://github.com/kubedb/project/issues/53). + +Run the following command to deploy the above `MongoDB` CRD object. + +```console +$ kubedb create -f https://raw.githubusercontent.com/kubedb/cli/0.8.0-beta.1/docs/examples/mongodb/monitoring/coreos-operator/demo-1.yaml +validating "https://raw.githubusercontent.com/kubedb/cli/0.8.0-beta.1/docs/examples/mongodb/monitoring/coreos-operator/demo-1.yaml" +mongodb "mgo-mon-coreos" created +``` + + +Here, + + - `spec.version` is the version of MongoDB database. In this tutorial, a MongoDB 3.4 database is going to be created. + + - `spec.storage` specifies the StorageClass of PVC dynamically allocated to store data for this database. This storage spec will be passed to the StatefulSet created by KubeDB operator to run database pods. You can specify any StorageClass available in your cluster with appropriate resource requests. If no storage spec is given, an `emptyDir` is used. + + - `spec.monitor` specifies that CoreOS Prometheus operator is used to monitor this database instance. A ServiceMonitor should be created in the `demo` namespace with label `app=kubedb`. The exporter endpoint should be scrapped every 10 seconds. + +KubeDB operator watches for `MongoDB` objects using Kubernetes api. When a `MongoDB` object is created, KubeDB operator will create a new StatefulSet and a ClusterIP Service with the matching crd name. KubeDB operator will also create a governing service for StatefulSets with the name `kubedb`, if one is not already present. + +```console +$ kubedb get mg -n demo +NAME STATUS AGE +mgo-mon-coreos Creating 36s + +$ kubedb get mg -n demo +NAME STATUS AGE +mgo-mon-coreos Running 1m + + +$ kubedb describe mg -n demo mgo-mon-coreos +Name: mgo-mon-coreos +Namespace: demo +StartTimestamp: Mon, 05 Feb 2018 11:20:20 +0600 +Status: Running +Volume: + StorageClass: standard + Capacity: 50Mi + Access Modes: RWO + +StatefulSet: + Name: mgo-mon-coreos + Replicas: 1 current / 1 desired + CreationTimestamp: Mon, 05 Feb 2018 11:20:27 +0600 + Pods Status: 1 Running / 0 Waiting / 0 Succeeded / 0 Failed + +Service: + Name: mgo-mon-coreos + Type: ClusterIP + IP: 10.107.145.36 + Port: db 27017/TCP + Port: prom-http 56790/TCP + +Database Secret: + Name: mgo-mon-coreos-auth + Type: Opaque + Data + ==== + password: 16 bytes + user: 4 bytes + +Monitoring System: + Agent: prometheus.io/coreos-operator + Prometheus: + Namespace: demo + Labels: app=kubedb + Interval: 10s + +No Snapshots. + +Events: + FirstSeen LastSeen Count From Type Reason Message + --------- -------- ----- ---- -------- ------ ------- + 10m 10m 1 MongoDB operator Normal Successful Successfully patched StatefulSet + 10m 10m 1 MongoDB operator Normal Successful Successfully patched MongoDB + 10m 10m 1 MongoDB operator Normal Successful Successfully created StatefulSet + 10m 10m 1 MongoDB operator Normal Successful Successfully created MongoDB + 11m 11m 1 MongoDB operator Normal Successful Successfully created Service +``` + + +Since `spec.monitoring` was configured, a ServiceMonitor object is created accordingly. You can verify it running the following commands: + +```yaml +$ kubectl get servicemonitor -n demo +NAME AGE +kubedb-demo-mgo-mon-coreos 11m + + + +$ kubectl get servicemonitor -n demo kubedb-demo-mgo-mon-coreos -o yaml +apiVersion: monitoring.coreos.com/v1 +kind: ServiceMonitor +metadata: + clusterName: "" + creationTimestamp: 2018-02-05T05:20:46Z + labels: + app: kubedb + monitoring.appscode.com/service: mgo-mon-coreos.demo + name: kubedb-demo-mgo-mon-coreos + namespace: demo + resourceVersion: "57754" + selfLink: /apis/monitoring.coreos.com/v1/namespaces/demo/servicemonitors/kubedb-demo-mgo-mon-coreos + uid: 5215258a-0a34-11e8-8d7f-080027c05a6e +spec: + endpoints: + - interval: 10s + path: /kubedb.com/v1alpha1/namespaces/demo/mongodbs/mgo-mon-coreos/metrics + port: prom-http + targetPort: 0 + namespaceSelector: + matchNames: + - demo + selector: + matchLabels: + kubedb.com/kind: MongoDB + kubedb.com/name: mgo-mon-coreos +``` + + +Now, if you go the Prometheus Dashboard, you should see that this database endpoint as one of the targets. + + +## Cleaning up +To cleanup the Kubernetes resources created by this tutorial, run: + +```console +$ kubedb delete mg,drmn,snap -n demo --all --force + +# In rbac enabled cluster, +# $ kubectl delete clusterrolebindings prometheus-operator prometheus +# $ kubectl delete clusterrole prometheus-operator prometheus + +$ kubectl delete ns demo +namespace "demo" deleted +``` + + +## Next Steps +- Monitor your MongoDB database with KubeDB using [out-of-the-box builtin-Prometheus](/docs/guides/mongodb/monitoring/using-builtin-prometheus.md). +- Detail concepts of [MongoDB object](/docs/concepts/databases/mongodb.md). +- [Snapshot and Restore](/docs/guides/mongodb/snapshot/backup-and-restore.md) process of MongoDB databases using KubeDB. +- Take [Scheduled Snapshot](/docs/guides/mongodb/snapshot/scheduled-backup.md) of MongoDB databases using KubeDB. +- Initialize [MongoDB with Script](/docs/guides/mongodb/initialization/using-script.md). +- Initialize [MongoDB with Snapshot](/docs/guides/mongodb/initialization/using-snapshot.md). +- Use [Private Docker Registry](/docs/guides/mongodb/private-registry/using-private-registry.md) to deploy MongoDB with KubeDB. +- Wondering what features are coming next? Please visit [here](/docs/roadmap.md). +- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/mongodb/overview.md b/docs/guides/mongodb/overview.md deleted file mode 100644 index 9debc3be1..000000000 --- a/docs/guides/mongodb/overview.md +++ /dev/null @@ -1,727 +0,0 @@ ---- -title: MongoDB -menu: - docs_0.8.0-beta.0: - identifier: guides-mongodb-overview - name: Overview - parent: guides-mongodb - weight: 10 -menu_name: docs_0.8.0-beta.0 -section_menu_id: guides -aliases: - - /docs/0.8.0-beta.0/guides/mongodb/ ---- - -> New to KubeDB? Please start [here](/docs/guides/README.md). - -# Running MongoDB -This tutorial will show you how to use KubeDB to run a MongoDB database. - -## Before You Begin -At first, you need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one by using [Minikube](https://github.com/kubernetes/minikube). - -Now, install KubeDB cli on your workstation and KubeDB operator in your cluster following the steps [here](/docs/setup/install.md). - -To keep things isolated, this tutorial uses a separate namespace called `demo` throughout this tutorial. This tutorial will also use a phpMyAdmin to connect and test MongoDB database, once it is running. Run the following command to prepare your cluster for this tutorial: - -```console -$ kubectl create -f ./docs/examples/mongodb/demo-0.yaml -namespace "demo" created - -$ kubectl get ns -NAME STATUS AGE -default Active 1h -demo Active 1m -kube-public Active 1h -kube-system Active 1h -``` - -## Create a MongoDB database -KubeDB implements a `MongoDB` CRD to define the specification of a MongoDB database. Below is the `MongoDB` object created in this tutorial. - -```yaml -apiVersion: kubedb.com/v1alpha1 -kind: MongoDB -metadata: - name: mgo1 - namespace: demo -spec: - version: 3.4 - doNotPause: true - storage: - storageClassName: "standard" - accessModes: - - ReadWriteOnce - resources: - requests: - storage: 50Mi - init: - scriptSource: - gitRepo: - repository: "https://github.com/kubedb/mongodb-init-scripts.git" - directory: . - - -$ kubedb create -f ./docs/examples/mongodb/demo-1.yaml -validating "./docs/examples/mongodb/demo-1.yaml" -mongodb "mg1" created -``` - -Here, - - `spec.version` is the version of MongoDB database. In this tutorial, a MongoDB 3.4 database is going to be created. - - - `spec.doNotPause` tells KubeDB operator that if this object is deleted, it should be automatically reverted. This should be set to true for production databases to avoid accidental deletion. - - - `spec.storage` specifies the StorageClass of PVC dynamically allocated to store data for this database. This storage spec will be passed to the StatefulSet created by KubeDB operator to run database pods. You can specify any StorageClass available in your cluster with appropriate resource requests. If no storage spec is given, an `emptyDir` is used. - - - `spec.init.scriptSource` specifies a sql script source used to initialize the database after it is created. The sql scripts will be executed alphabatically. In this tutorial, a sample sql script from the git repository `https://github.com/kubedb/mongodb-init-scripts.git` is used to create a test database. - -KubeDB operator watches for `MongoDB` objects using Kubernetes api. When a `MongoDB` object is created, KubeDB operator will create a new StatefulSet and a ClusterIP Service with the matching MongoDB object name. KubeDB operator will also create a governing service for StatefulSets with the name `kubedb`, if one is not already present. If [RBAC is enabled](/docs/guides/rbac.md), a ClusterRole, ServiceAccount and ClusterRoleBinding with the matching object name will be created and used as the service account name for the corresponding StatefulSet. - -```console -$ kubedb describe mg -n demo mgo1 -Name: mgo1 -Namespace: demo -StartTimestamp: Mon, 11 Dec 2017 12:38:54 +0600 -Status: Running -Volume: - StorageClass: standard - Capacity: 50Mi - Access Modes: RWO - -StatefulSet: - Name: mgo1 - Replicas: 1 current / 1 desired - CreationTimestamp: Mon, 11 Dec 2017 12:38:58 +0600 - Pods Status: 1 Running / 0 Waiting / 0 Succeeded / 0 Failed - -Service: - Name: mgo1 - Type: ClusterIP - IP: 10.105.200.75 - Port: db 27017/TCP - -Database Secret: - Name: mgo1-admin-auth - Type: Opaque - Data - ==== - .admin: 16 bytes - -No Snapshots. - -Events: - FirstSeen LastSeen Count From Type Reason Message - --------- -------- ----- ---- -------- ------ ------- - 1m 1m 1 MongoDB operator Normal SuccessfulValidate Successfully validate MongoDB - 1m 1m 1 MongoDB operator Normal SuccessfulValidate Successfully validate MongoDB - 1m 1m 1 MongoDB operator Normal SuccessfulCreate Successfully created StatefulSet - 1m 1m 1 MongoDB operator Normal SuccessfulCreate Successfully created MongoDB - 5m 5m 1 MongoDB operator Normal SuccessfulValidate Successfully validate MongoDB - 5m 5m 1 MongoDB operator Normal Creating Creating Kubernetes objects - - -$ kubectl get statefulset -n demo -NAME DESIRED CURRENT AGE -mgo1 1 1 6m - - -$ kubectl get pvc -n demo -NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE -data-mgo1-0 Bound pvc-f7746b65-de3d-11e7-879f-0800279fc284 50Mi RWO standard 6m - -$ kubectl get pv -n demo -NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE -pvc-f7746b65-de3d-11e7-879f-0800279fc284 50Mi RWO Delete Bound demo/data-mgo1-0 standard 6m - -$ kubectl get service -n demo -NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE -kubedb ClusterIP None 7m -mgo1 ClusterIP 10.105.200.75 27017/TCP 7m -``` - -KubeDB operator sets the `status.phase` to `Running` once the database is successfully created. Run the following command to see the modified MongoDB object: - -```yaml -$ kubedb get mg -n demo mgo1 -o yaml -apiVersion: kubedb.com/v1alpha1 -kind: MongoDB -metadata: - clusterName: "" - creationTimestamp: 2017-12-11T06:38:54Z - deletionGracePeriodSeconds: null - deletionTimestamp: null - generation: 0 - initializers: null - name: mgo1 - namespace: demo - resourceVersion: "717" - selfLink: /apis/kubedb.com/v1alpha1/namespaces/demo/mongodbs/mgo1 - uid: f52ab165-de3d-11e7-879f-0800279fc284 -spec: - doNotPause: true - databaseSecret: - secretName: mgo1-admin-auth - init: - scriptSource: - gitRepo: - directory: . - repository: https://github.com/kubedb/mongodb-init-scripts.git - storage: - accessModes: - - ReadWriteOnce - resources: - requests: - storage: 50Mi - storageClassName: standard - version: 3.4 -status: - creationTime: 2017-12-11T06:38:54Z - phase: Running -``` - -Please note that KubeDB operator has created a new Secret called `mgo1-admin-auth` (format: {mongodb-object-name}-admin-auth) for storing the password for `mongodb` superuser. This secret contains a `.admin` key which contains the password for `mongodb` superuser. If you want to use an existing secret please specify that when creating the MongoDB object using `spec.databaseSecret.secretName`. - -Now, you can connect to this database through [mongo-shell](https://docs.mongodb.com/v3.4/mongo/). In this tutorial, we are connecting to the MongoDB server from inside of pod. -```console -$ kubectl get secrets -n demo mgo1-admin-auth -o jsonpath='{.data.\.admin}' | base64 -d -aaqCftpLsaGDLVIo - -$ kubectl exec -it mgo1-0 -n demo sh - -> mongo admin -MongoDB shell version v3.4.10 -connecting to: mongodb://127.0.0.1:27017/admin -MongoDB server version: 3.4.10 -Welcome to the MongoDB shell. -For interactive help, type "help". -For more comprehensive documentation, see - http://docs.mongodb.org/ -Questions? Try the support group - http://groups.google.com/group/mongodb-user - -> db.auth("root","aaqCftpLsaGDLVIo") -1 - -> show dbs -admin 0.000GB -local 0.000GB -mydb 0.000GB - -> show users -{ - "_id" : "admin.root", - "user" : "root", - "db" : "admin", - "roles" : [ - { - "role" : "root", - "db" : "admin" - } - ] -} - -> use newdb -switched to db newdb - -> db.movie.insert({"name":"batman"}); -WriteResult({ "nInserted" : 1 }) - -> db.movie.find().pretty() -{ "_id" : ObjectId("5a2e435d7ec14e7bda785f16"), "name" : "batman" } - -> exit -bye -``` - - -## Database Snapshots - -### Instant Backups -Now, you can easily take a snapshot of this database by creating a `Snapshot` object. When a `Snapshot` object is created, KubeDB operator will launch a Job that runs the `mongodump` command and uploads the output bson file to various cloud providers S3, GCS, Azure, OpenStack Swift and/or locally mounted volumes using [osm](https://github.com/appscode/osm). - -In this tutorial, snapshots will be stored in a Google Cloud Storage (GCS) bucket. To do so, a secret is needed that has the following 2 keys: - -| Key | Description | -|-----------------------------------|------------------------------------------------------------| -| `GOOGLE_PROJECT_ID` | `Required`. Google Cloud project ID | -| `GOOGLE_SERVICE_ACCOUNT_JSON_KEY` | `Required`. Google Cloud service account json key | - -```console -$ echo -n '' > GOOGLE_PROJECT_ID -$ mv downloaded-sa-json.key > GOOGLE_SERVICE_ACCOUNT_JSON_KEY -$ kubectl create secret generic mg-snap-secret -n demo \ - --from-file=./GOOGLE_PROJECT_ID \ - --from-file=./GOOGLE_SERVICE_ACCOUNT_JSON_KEY -secret "mg-snap-secret" created -``` - -```yaml -$ kubectl get secret mg-snap-secret -n demo -o yaml -apiVersion: v1 -data: - GOOGLE_PROJECT_ID: PHlvdXItcHJvamVjdC1pZD4= - GOOGLE_SERVICE_ACCOUNT_JSON_KEY: ewogICJ0eXBlIjogInNlcnZpY2VfYWNjb3V...9tIgp9Cg== -kind: Secret -metadata: - creationTimestamp: 2017-12-11T08:40:34Z - name: mg-snap-secret - namespace: demo - resourceVersion: "4140" - selfLink: /api/v1/namespaces/demo/secrets/mg-snap-secret - uid: f44826c4-de4e-11e7-879f-0800279fc284 -type: Opaque -``` - -To lean how to configure other storage destinations for Snapshots, please visit [here](/docs/concepts/snapshot.md). Now, create the Snapshot object. - -```console -$ kubedb create -f ./docs/examples/mongodb/demo-2.yaml -validating "./docs/examples/mongodb/demo-2.yaml" -snapshot "mgo-xyz" created - -$ kubedb get snap -n demo -NAME DATABASE STATUS AGE -mgo-xyz mg/mgo1 Running 20s -``` - -```yaml -$ kubedb get snap -n demo mgo-xyz -o yaml -apiVersion: kubedb.com/v1alpha1 -kind: Snapshot -metadata: - clusterName: "" - creationTimestamp: 2017-12-11T08:41:54Z - deletionGracePeriodSeconds: null - deletionTimestamp: null - generation: 0 - initializers: null - labels: - kubedb.com/kind: MongoDB - kubedb.com/name: mgo1 - snapshots.kubedb.com/status: Running - name: mgo-xyz - namespace: demo - resourceVersion: "4243" - selfLink: /apis/kubedb.com/v1alpha1/namespaces/demo/snapshots/mgo-xyz - uid: 2421bd0a-de4f-11e7-879f-0800279fc284 -spec: - databaseName: mgo1 - gcs: - bucket: restic - storageSecretName: mg-snap-secret -status: - phase: Running - startTime: 2017-12-11T08:41:54Z -``` - -Here, - -- `metadata.labels` should include the type of database `kubedb.com/kind: MongoDB` whose snapshot will be taken. - -- `spec.databaseName` points to the database whose snapshot is taken. - -- `spec.storageSecretName` points to the Secret containing the credentials for snapshot storage destination. - -- `spec.gcs.bucket` points to the bucket name used to store the snapshot data. - - -You can also run the `kubedb describe` command to see the recent snapshots taken for a database. - -```console -$ kubedb describe mg -n demo mgo1 -Name: mgo1 -Namespace: demo -StartTimestamp: Mon, 11 Dec 2017 12:38:54 +0600 -Status: Running -Volume: - StorageClass: standard - Capacity: 50Mi - Access Modes: RWO - -StatefulSet: - Name: mgo1 - Replicas: 1 current / 1 desired - CreationTimestamp: Mon, 11 Dec 2017 12:38:58 +0600 - Pods Status: 1 Running / 0 Waiting / 0 Succeeded / 0 Failed - -Service: - Name: mgo1 - Type: ClusterIP - IP: 10.105.200.75 - Port: db 27017/TCP - -Database Secret: - Name: mgo1-admin-auth - Type: Opaque - Data - ==== - .admin: 16 bytes - -Snapshots: - Name Bucket StartTime CompletionTime Phase - ---- ------ --------- -------------- ----- - mgo-xyz gs:restic Mon, 11 Dec 2017 14:41:54 +0600 Mon, 11 Dec 2017 14:48:58 +0600 Succeeded - -Events: - FirstSeen LastSeen Count From Type Reason Message - --------- -------- ----- ---- -------- ------ ------- - 2m 2m 1 Snapshot Controller Normal SuccessfulSnapshot Successfully completed snapshot - 9m 9m 1 Snapshot Controller Normal Starting Backup running -``` - -Once the snapshot Job is complete, you should see the output of the `mongodump` command stored in the GCS bucket. - -![snapshot-console](/docs/images/mongodb/mgo-xyz-snapshot.png) - -From the above image, you can see that the snapshot output is stored in a folder called `{bucket}/kubedb/{namespace}/{mongodb-object}/{snapshot}/`. - - -### Scheduled Backups -KubeDB supports taking periodic backups for a database using a [cron expression](https://github.com/robfig/cron/blob/v2/doc.go#L26). To take periodic backups, edit the MongoDB to add `spec.backupSchedule` section. - -```yaml -$ kubedb edit mg mgo1 -n demo -apiVersion: kubedb.com/v1alpha1 -kind: MongoDB -metadata: - clusterName: "" - creationTimestamp: 2017-12-11T06:38:54Z - generation: 0 - initializers: null - name: mgo1 - namespace: demo - resourceVersion: "5378" - selfLink: /apis/kubedb.com/v1alpha1/namespaces/demo/mongodbs/mgo1 - uid: f52ab165-de3d-11e7-879f-0800279fc284 -spec: - doNotPause: true - databaseSecret: - secretName: mgo1-admin-auth - init: - scriptSource: - gitRepo: - directory: . - repository: https://github.com/kubedb/mongodb-init-scripts.git - storage: - accessModes: - - ReadWriteOnce - resources: - requests: - storage: 50Mi - storageClassName: standard - version: 3.4 - backupSchedule: - cronExpression: '@every 1m' - gcs: - bucket: restic - storageSecretName: mg-snap-secret -status: - creationTime: 2017-12-11T06:38:54Z - phase: Running -``` - -Once the `spec.backupSchedule` is added, KubeDB operator will create a new Snapshot object on each tick of the cron expression. This triggers KubeDB operator to create a Job as it would for any regular instant backup process. You can see the snapshots as they are created using `kubedb get snap` command. -```console -$ kubedb get snap -n demo -NAME DATABASE STATUS AGE -mgo-xyz mg/mgo1 Succeeded 21m -mgo1-20171211-090039 mg/mgo1 Succeeded 2m -mgo1-20171211-090159 mg/mgo1 Succeeded 1m -mgo1-20171211-090259 mg/mgo1 Running 3s -``` - -### Restore from Snapshot -You can create a new database from a previously taken Snapshot. Specify the Snapshot name in the `spec.init.snapshotSource` field of a new MongoDB object. See the example `recovered` object below: - -```yaml -$ cat ./docs/examples/mongodb/demo-4.yaml -apiVersion: kubedb.com/v1alpha1 -kind: MongoDB -metadata: - name: recovered - namespace: demo -spec: - version: 3.4 - storage: - storageClassName: "standard" - accessModes: - - ReadWriteOnce - resources: - requests: - storage: 50Mi - init: - snapshotSource: - name: mgo-xyz - - -$ kubedb create -f ./docs/examples/mongodb/demo-4.yaml -validating "./docs/examples/mongodb/demo-4.yaml" -mongodb "recovered" created -``` - -Here, - - `spec.init.snapshotSource.name` refers to a Snapshot object for a MongoDB database in the same namespaces as this new `recovered` MongoDB object. - -Now, wait several seconds. KubeDB operator will create a new StatefulSet. Then KubeDB operator launches a Kubernetes Job to initialize the new database using the data from `mgo-xyz` Snapshot. - -```console -$ kubedb get mg -n demo -NAME STATUS AGE -mgo1 Running 2h -recovered Running 1m - - -$ kubedb describe mg -n demo recovered -Name: recovered -Namespace: demo -StartTimestamp: Mon, 11 Dec 2017 15:04:34 +0600 -Status: Running -Volume: - StorageClass: standard - Capacity: 50Mi - Access Modes: RWO - -StatefulSet: - Name: recovered - Replicas: 1 current / 1 desired - CreationTimestamp: Mon, 11 Dec 2017 15:04:38 +0600 - Pods Status: 1 Running / 0 Waiting / 0 Succeeded / 0 Failed - -Service: - Name: recovered - Type: ClusterIP - IP: 10.103.115.39 - Port: db 27017/TCP - -Database Secret: - Name: recovered-admin-auth - Type: Opaque - Data - ==== - .admin: 16 bytes - -No Snapshots. - -Events: - FirstSeen LastSeen Count From Type Reason Message - --------- -------- ----- ---- -------- ------ ------- - 1m 1m 1 MongoDB operator Normal SuccessfulValidate Successfully validate MongoDB - 1m 1m 1 MongoDB operator Normal SuccessfulValidate Successfully validate MongoDB - 1m 1m 1 MongoDB operator Normal SuccessfulInitialize Successfully completed initialization - 1m 1m 1 MongoDB operator Normal SuccessfulCreate Successfully created MongoDB - 1m 1m 1 MongoDB operator Normal SuccessfulCreate Successfully created StatefulSet - 1m 1m 1 MongoDB operator Normal Initializing Initializing from Snapshot: "mgo-xyz" - 2m 2m 1 MongoDB operator Normal SuccessfulValidate Successfully validate MongoDB - 2m 2m 1 MongoDB operator Normal Creating Creating Kubernetes objects -``` - -## Pause Database - -Since the MongoDB object created in this tutorial has `spec.doNotPause` set to true, if you delete the MongoDB object, KubeDB operator will recreate the object and essentially nullify the delete operation. You can see this below: - -```console -$ kubedb delete mg mgo1 -n demo -error: MongoDB "mgo1" can't be paused. To continue delete, unset spec.doNotPause and retry. -``` - -Now, run `kubedb edit mg mgo1 -n demo` to set `spec.doNotPause` to false or remove this field (which default to false). Then if you delete the MongoDB object, KubeDB operator will delete the StatefulSet and its pods, but leaves the PVCs unchanged. In KubeDB parlance, we say that `mgo1` MongoDB database has entered into dormant state. This is represented by KubeDB operator by creating a matching DormantDatabase object. - -```yaml -$ kubedb delete mg mgo1 -n demo -mongodb "mgo1" deleted - -$ kubedb get drmn -n demo mgo1 -NAME STATUS AGE -mgo1 Pausing 39s - -$ kubedb get drmn -n demo mgo1 -NAME STATUS AGE -mgo1 Paused 1m - - -$ kubedb get drmn -n demo mgo1 -o yaml -apiVersion: kubedb.com/v1alpha1 -kind: DormantDatabase -metadata: - annotations: - mongodbs.kubedb.com/init: '{"scriptSource":{"gitRepo":{"repository":"https://github.com/kubedb/mongodb-init-scripts.git","directory":"."}}}' - clusterName: "" - creationTimestamp: 2017-12-11T09:17:01Z - deletionGracePeriodSeconds: null - deletionTimestamp: null - generation: 0 - initializers: null - labels: - kubedb.com/kind: MongoDB - name: mgo1 - namespace: demo - resourceVersion: "7029" - selfLink: /apis/kubedb.com/v1alpha1/namespaces/demo/dormantdatabases/mgo1 - uid: 0ba4a0df-de54-11e7-879f-0800279fc284 -spec: - origin: - metadata: - creationTimestamp: null - name: mgo1 - namespace: demo - spec: - mongodb: - databaseSecret: - secretName: mgo1-admin-auth - resources: {} - storage: - accessModes: - - ReadWriteOnce - resources: - requests: - storage: 50Mi - storageClassName: standard - version: "3.4" -status: - creationTime: 2017-12-11T09:17:01Z - pausingTime: 2017-12-11T09:18:11Z - phase: Paused -``` - -Here, - - `spec.origin` is the spec of the original spec of the original MongoDB object. - - - `status.phase` points to the current database state `Paused`. - - -## Resume Dormant Database - -To resume the database from the dormant state, set `spec.resume` to `true` in the DormantDatabase object. - -```yaml -$ kubedb edit drmn -n demo mgo1 -apiVersion: kubedb.com/v1alpha1 -kind: DormantDatabase -metadata: - annotations: - mongodbs.kubedb.com/init: '{"scriptSource":{"gitRepo":{"repository":"https://github.com/kubedb/mongodb-init-scripts.git","directory":"."}}}' - clusterName: "" - creationTimestamp: 2017-12-11T09:17:01Z - deletionGracePeriodSeconds: null - deletionTimestamp: null - generation: 0 - initializers: null - labels: - kubedb.com/kind: MongoDB - name: mgo1 - namespace: demo - resourceVersion: "7029" - selfLink: /apis/kubedb.com/v1alpha1/namespaces/demo/dormantdatabases/mgo1 - uid: 0ba4a0df-de54-11e7-879f-0800279fc284 -spec: - resume: true - origin: - metadata: - creationTimestamp: null - name: mgo1 - namespace: demo - spec: - mongodb: - databaseSecret: - secretName: mgo1-admin-auth - resources: {} - storage: - accessModes: - - ReadWriteOnce - resources: - requests: - storage: 50Mi - storageClassName: standard - version: "3.4" -status: - creationTime: 2017-12-11T09:17:01Z - pausingTime: 2017-12-11T09:18:11Z - phase: Paused - -``` - -KubeDB operator will notice that `spec.resume` is set to true. KubeDB operator will delete the DormantDatabase object and create a new MongoDB object using the original spec. This will in turn start a new StatefulSet which will mount the originally created PVCs. Thus the original database is resumed. - -## Wipeout Dormant Database -You can also wipe out a DormantDatabase by setting `spec.wipeOut` to true. KubeDB operator will delete the PVCs, delete any relevant Snapshot objects for this database and also delete snapshot data stored in the Cloud Storage buckets. There is no way to resume a wiped out database. So, be sure before you wipe out a database. - -```yaml -$ kubedb edit drmn -n demo mgo1 -# set spec.wipeOut: true - -$ kubedb get drmn -n demo mgo1 -o yaml -apiVersion: kubedb.com/v1alpha1 -kind: DormantDatabase -metadata: - clusterName: "" - creationTimestamp: 2017-12-11T09:22:11Z - generation: 0 - initializers: null - labels: - kubedb.com/kind: MongoDB - name: mgo1 - namespace: demo - resourceVersion: "7497" - selfLink: /apis/kubedb.com/v1alpha1/namespaces/demo/dormantdatabases/mgo1 - uid: c4b9cb9c-de54-11e7-879f-0800279fc284 -spec: - origin: - metadata: - annotations: - mongodbs.kubedb.com/init: '{"scriptSource":{"gitRepo":{"repository":"https://github.com/kubedb/mongodb-init-scripts.git","directory":"."}}}' - creationTimestamp: null - name: mgo1 - namespace: demo - spec: - mongodb: - databaseSecret: - secretName: mgo1-admin-auth - resources: {} - storage: - accessModes: - - ReadWriteOnce - resources: - requests: - storage: 50Mi - storageClassName: standard - version: "3.4" - wipeOut: true -status: - creationTime: 2017-12-11T09:22:11Z - pausingTime: 2017-12-11T09:23:11Z - phase: WipedOut - wipeOutTime: 2017-12-11T09:24:09Z - - -$ kubedb get drmn -n demo -NAME STATUS AGE -mgo1 WipedOut 3m -``` - - -## Delete Dormant Database -You still have a record that there used to be a MongoDB database `mgo1` in the form of a DormantDatabase database `mgo1`. Since you have already wiped out the database, you can delete the DormantDatabase object. - -```console -$ kubedb delete drmn mgo1 -n demo -dormantdatabase "mgo1" deleted -``` - -## Cleaning up -To cleanup the Kubernetes resources created by this tutorial, run: -```console -$ kubectl delete ns demo -namespace "demo" deleted -``` - -If you would like to uninstall KubeDB operator, please follow the steps [here](/docs/setup/uninstall.md). - - -## Next Steps -- Learn about the details of MongoDB object [here](/docs/concepts/databases/mongodb.md). -- See the list of supported storage providers for snapshots [here](/docs/concepts/snapshot.md). -- Thinking about monitoring your database? KubeDB works [out-of-the-box with Prometheus](/docs/guides/monitoring.md). -- Learn how to use KubeDB in a [RBAC](/docs/guides/rbac.md) enabled cluster. -- Wondering what features are coming next? Please visit [here](/docs/roadmap.md). -- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/mongodb/private-registry/using-private-registry.md b/docs/guides/mongodb/private-registry/using-private-registry.md new file mode 100644 index 000000000..f1d738c57 --- /dev/null +++ b/docs/guides/mongodb/private-registry/using-private-registry.md @@ -0,0 +1,142 @@ +> New to KubeDB? Please start [here](/docs/guides/README.md). + +# Using Private Docker Registry + +KubeDB operator supports using private Docker registry. This tutorial will show you how to use KubeDB to run MongoDB database using private Docker images. + +## Before You Begin +At first, you need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one by using [Minikube](https://github.com/kubernetes/minikube). + +You will also need a docker private [registry](https://docs.docker.com/registry/) or [private repository](https://docs.docker.com/docker-hub/repos/#private-repositories). In this tutorial we will use private repository of [docker hub](https://hub.docker.com/). + +You have to push the required images from KubeDB's [Docker hub account](https://hub.docker.com/r/kubedb/) into your private registry. For mongodb, push the following images to your private registry. + + - [kubedb/operator](https://hub.docker.com/r/kubedb/operator) + - [kubedb/mongo](https://hub.docker.com/r/kubedb/mongo) + - [kubedb/mongo-tools](https://hub.docker.com/r/kubedb/mongo-tools) + +```console +$ export DOCKER_REGISTRY= + +$ docker pull kubedb/operator:0.8.0-beta.0-4 ; docker tag kubedb/operator:0.8.0-beta.0-4 $DOCKER_REGISTRY/operator:0.8.0-beta.0-4 ; docker push $DOCKER_REGISTRY/operator:0.8.0-beta.0-4 +$ docker pull kubedb/mongo:3.4 ; docker tag kubedb/mongo:3.4 $DOCKER_REGISTRY/mongo:3.4 ; docker push $DOCKER_REGISTRY/mongo:3.4 +$ docker pull kubedb/mongo:3.6 ; docker tag kubedb/mongo:3.6 $DOCKER_REGISTRY/mongo:3.6 ; docker push $DOCKER_REGISTRY/mongo:3.6 +$ docker pull kubedb/mongo-tools:3.4 ; docker tag kubedb/mongo-tools:3.4 $DOCKER_REGISTRY/mongo-tools:3.4 ; docker push $DOCKER_REGISTRY/mongo-tools:3.4 +$ docker pull kubedb/mongo-tools:3.6 ; docker tag kubedb/mongo-tools:3.6 $DOCKER_REGISTRY/mongo-tools:3.6 ; docker push $DOCKER_REGISTRY/mongo-tools:3.6 +``` + +## Create ImagePullSecret + +ImagePullSecrets is a type of a Kubernete Secret whose sole purpose is to pull private images from a Docker registry. It allows you to specify the url of the docker registry, credentials for logging in and the image name of your private docker image. + +Run the following command, substituting the appropriate uppercase values to create an image pull secret for your private docker registry: + +```console +$ kubectl create secret docker-registry myregistrykey \ + --docker-server=DOCKER_REGISTRY_SERVER \ + --docker-username=DOCKER_USER \ + --docker-email=DOCKER_EMAIL \ + --docker-password=DOCKER_PASSWORD + +secret "myregistrykey" created. +``` + +If you wish to follow other ways to pull private images see [official docs](https://kubernetes.io/docs/concepts/containers/images/) of kubernetes. + + +## Install KubeDB operator + +When installing KubeDB operator, set the flags `--docker-registry` and `--image-pull-secret` to appropriate value. Follow the steps to [install KubeDB operator](/docs/setup/install.md) properly in cluster so that to points to the DOCKER_REGISTRY you wish to pull images from. + +## Create Demo namespace + +To keep things isolated, this tutorial uses a separate namespace called `demo` throughout this tutorial. Run the following command to prepare your cluster for this tutorial: + +```console +$ kubectl create -f https://raw.githubusercontent.com/kubedb/cli/0.8.0-beta.1/docs/examples/mongodb/demo-0.yaml +namespace "demo" created + +$ kubectl get ns +NAME STATUS AGE +default Active 45m +demo Active 10s +kube-public Active 45m +kube-system Active 45m +``` + +## Deploy MongoDB database from Private Registry + +While deploying `MongoDB` from private repository, you have to add `myregistrykey` secret in `MongoDB` `spec.imagePullSecrets`. +Below is the MongoDB CRD object we will create. + +```yaml +apiVersion: kubedb.com/v1alpha1 +kind: MongoDB +metadata: + name: mgo-pvt-reg + namespace: demo +spec: + version: 3.4 + doNotPause: true + storage: + storageClassName: "standard" + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 50Mi + imagePullSecrets: + - name: myregistrykey +``` + +Now run the command to deploy this `MongoDB` object: + +```console +$ kubedb create -f https://raw.githubusercontent.com/kubedb/cli/0.8.0-beta.1/docs/examples/mongodb/private-registry/demo-2.yaml +validating "https://raw.githubusercontent.com/kubedb/cli/0.8.0-beta.1/docs/examples/mongodb/private-registry/demo-2.yaml" +mongodb "mgo-pvt-reg" created +``` + +To check if the images pulled successfully from the repository, see if the `MongoDB` is in running state: + +```console +$ kubectl get pods -n demo -w +NAME READY STATUS RESTARTS AGE +mgo-pvt-reg-0 0/1 Pending 0 0s +mgo-pvt-reg-0 0/1 Pending 0 0s +mgo-pvt-reg-0 0/1 ContainerCreating 0 0s +mgo-pvt-reg-0 1/1 Running 0 5m + + +$ kubedb get mg -n demo +NAME STATUS AGE +mgo-pvt-reg Running 1m +``` + +## Snapshot + +We don't need to add `imagePullSecret` for `snapshot` objects. +Just create [snapshot object](/docs/guides/mongodb/snapshot/backup-and-restore.md) and KubeDB operator will reuse the `ImagePullSecret` from `MongoDB` object. + +## Cleaning up +To cleanup the Kubernetes resources created by this tutorial, run: + +```console +$ kubedb delete mg,drmn,snap -n demo --all --force + +$ kubectl delete ns demo +namespace "demo" deleted +``` + + +## Next Steps +- [Snapshot and Restore](/docs/guides/mongodb/snapshot/backup-and-restore.md) process of MongoDB databases using KubeDB. +- Take [Scheduled Snapshot](/docs/guides/mongodb/snapshot/scheduled-backup.md) of MongoDB databases using KubeDB. +- Initialize [MongoDB with Script](/docs/guides/mongodb/initialization/using-script.md). +- Initialize [MongoDB with Snapshot](/docs/guides/mongodb/initialization/using-snapshot.md). +- Monitor your MongoDB database with KubeDB using [out-of-the-box CoreOS Prometheus Operator](/docs/guides/mongodb/monitoring/using-coreos-prometheus-operator.md). +- Monitor your MongoDB database with KubeDB using [out-of-the-box builtin-Prometheus](/docs/guides/mongodb/monitoring/using-builtin-prometheus.md). +- Detail concepts of [MongoDB object](/docs/concepts/databases/mongodb.md). +- Detail concepts of [Snapshot object](/docs/concepts/snapshot.md). +- Wondering what features are coming next? Please visit [here](/docs/roadmap.md). +- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/mongodb/quickstart/quickstart.md b/docs/guides/mongodb/quickstart/quickstart.md new file mode 100644 index 000000000..14b81edbb --- /dev/null +++ b/docs/guides/mongodb/quickstart/quickstart.md @@ -0,0 +1,422 @@ +> New to KubeDB? Please start [here](/docs/guides/README.md). + +# MongoDB QuickStart +This tutorial will show you how to use KubeDB to run a MongoDB database. + +

+  lifecycle +

+ +## Before You Begin +At first, you need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one by using [Minikube](https://github.com/kubernetes/minikube). + +Now, install KubeDB cli on your workstation and KubeDB operator in your cluster following the steps [here](/docs/setup/install.md). + +To keep things isolated, this tutorial uses a separate namespace called `demo` throughout this tutorial. Run the following command to prepare your cluster for this tutorial: + +```console +$ kubectl create -f https://raw.githubusercontent.com/kubedb/cli/0.8.0-beta.1/docs/examples/mongodb/demo-0.yaml +namespace "demo" created + +$ kubectl get ns +NAME STATUS AGE +default Active 45m +demo Active 10s +kube-public Active 45m +kube-system Active 45m +``` + +Please note that the yaml files that are used in this tutorial, stored in [docs/examples](https://github.com/kubedb/cli/tree/master/docs/examples) folder in GitHub repository [kubedb/cli](https://github.com/kubedb/cli). + +## Create a MongoDB database +KubeDB implements a `MongoDB` CRD to define the specification of a MongoDB database. Below is the `MongoDB` object created in this tutorial. + +```yaml +apiVersion: kubedb.com/v1alpha1 +kind: MongoDB +metadata: + name: mgo-quickstart + namespace: demo +spec: + version: 3.4 + doNotPause: true + storage: + storageClassName: "standard" + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 50Mi +``` + +```console +$ kubedb create -f https://raw.githubusercontent.com/kubedb/cli/0.8.0-beta.1/docs/examples/mongodb/quickstart/demo-1.yaml +validating "https://raw.githubusercontent.com/kubedb/cli/0.8.0-beta.1/docs/examples/mongodb/quickstart/demo-1.yaml" +mongodb "mgo-quickstart" created +``` + + +Here, + + - `spec.version` is the version of MongoDB database. In this tutorial, a MongoDB 3.4 database is going to be created. + + - `spec.doNotPause` tells KubeDB operator that if this object is deleted, it should be automatically reverted. This should be set to true for production databases to avoid accidental deletion. + + - `spec.storage` specifies the StorageClass of PVC dynamically allocated to store data for this database. This storage spec will be passed to the StatefulSet created by KubeDB operator to run database pods. You can specify any StorageClass available in your cluster with appropriate resource requests. If no storage spec is given, an `emptyDir` is used. + +KubeDB operator watches for `MongoDB` objects using Kubernetes api. When a `MongoDB` object is created, KubeDB operator will create a new StatefulSet and a ClusterIP Service with the matching MongoDB object name. KubeDB operator will also create a governing service for StatefulSets with the name `kubedb`, if one is not already present. No MongoDB specific RBAC permission is required in [RBAC enabled clusters](/docs/guides/rbac.md). + +```console +$ kubedb describe mg -n demo mgo-quickstart +Name: mgo-quickstart +Namespace: demo +StartTimestamp: Fri, 02 Feb 2018 15:11:58 +0600 +Status: Running +Volume: + StorageClass: standard + Capacity: 50Mi + Access Modes: RWO + +StatefulSet: + Name: mgo-quickstart + Replicas: 1 current / 1 desired + CreationTimestamp: Fri, 02 Feb 2018 15:11:24 +0600 + Pods Status: 1 Running / 0 Waiting / 0 Succeeded / 0 Failed + +Service: + Name: mgo-quickstart + Type: ClusterIP + IP: 10.103.114.139 + Port: db 27017/TCP + +Database Secret: + Name: mgo-quickstart-auth + Type: Opaque + Data + ==== + password: 16 bytes + user: 4 bytes + +No Snapshots. + +Events: + FirstSeen LastSeen Count From Type Reason Message + --------- -------- ----- ---- -------- ------ ------- + 2m 2m 1 MongoDB operator Normal Successful Successfully patched StatefulSet + 2m 2m 1 MongoDB operator Normal Successful Successfully patched MongoDB + 2m 2m 1 MongoDB operator Normal Successful Successfully patched StatefulSet + 2m 2m 1 MongoDB operator Normal Successful Successfully patched MongoDB + + +$ kubectl get statefulset -n demo +NAME DESIRED CURRENT AGE +mgo-quickstart 1 1 4m + + +$ kubectl get pvc -n demo +NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE +data-mgo-quickstart-0 Bound pvc-16158aae-07fa-11e8-946f-080027c05a6e 50Mi RWO standard 2m + + +$ kubectl get pv -n demo +NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE +pvc-16158aae-07fa-11e8-946f-080027c05a6e 50Mi RWO Delete Bound demo/data-mgo-quickstart-0 standard 3m + + +$ kubectl get service -n demo +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +kubedb ClusterIP None 3m +mgo-quickstart ClusterIP 10.107.133.189 27017/TCP 3m +``` + + +KubeDB operator sets the `status.phase` to `Running` once the database is successfully created. Run the following command to see the modified MongoDB object: + +```yaml +$ kubedb get mg -n demo mgo-quickstart -o yaml +apiVersion: kubedb.com/v1alpha1 +kind: MongoDB +metadata: + clusterName: "" + creationTimestamp: 2018-02-02T09:18:39Z + finalizers: + - kubedb.com + generation: 0 + name: mgo-quickstart + namespace: demo + resourceVersion: "46856" + selfLink: /apis/kubedb.com/v1alpha1/namespaces/demo/mongodbs/mgo-quickstart + uid: 0de4d2a2-07fa-11e8-946f-080027c05a6e +spec: + databaseSecret: + secretName: mgo-quickstart-auth + doNotPause: true + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 50Mi + storageClassName: standard + version: 3.4 +status: + creationTime: 2018-02-02T09:18:50Z + phase: Running +``` + +Please note that KubeDB operator has created a new Secret called `mgo-quickstart-auth` *(format: {mongodb-object-name}-auth)* for storing the password for `mongodb` superuser. This secret contains a `user` key which contains the *username* for MongoDB superuser and a `password` key which contains the *password* for MongoDB superuser. + If you want to use an existing secret please specify that when creating the MongoDB object using `spec.databaseSecret.secretName`. While creating this secret manually, make sure the secret contains these two keys containing data `user` and `password`. + +Now, you can connect to this database through [mongo-shell](https://docs.mongodb.com/v3.4/mongo/). In this tutorial, we are connecting to the MongoDB server from inside the pod. + +```console +$ kubectl get secrets -n demo mgo-quickstart-auth -o jsonpath='{.data.\user}' | base64 -d +root + +$ kubectl get secrets -n demo mgo-quickstart-auth -o jsonpath='{.data.\password}' | base64 -d +aaqCftpLsaGDLVIo + +$ kubectl exec -it mgo-quickstart-0 -n demo sh + +> mongo admin +MongoDB shell version v3.4.10 +connecting to: mongodb://127.0.0.1:27017/admin +MongoDB server version: 3.4.10 +Welcome to the MongoDB shell. +For interactive help, type "help". +For more comprehensive documentation, see + http://docs.mongodb.org/ +Questions? Try the support group + http://groups.google.com/group/mongodb-user + +> db.auth("root","aaqCftpLsaGDLVIo") +1 + +> show dbs +admin 0.000GB +local 0.000GB +mydb 0.000GB + +> show users +{ + "_id" : "admin.root", + "user" : "root", + "db" : "admin", + "roles" : [ + { + "role" : "root", + "db" : "admin" + } + ] +} + +> use newdb +switched to db newdb + +> db.movie.insert({"name":"batman"}); +WriteResult({ "nInserted" : 1 }) + +> db.movie.find().pretty() +{ "_id" : ObjectId("5a2e435d7ec14e7bda785f16"), "name" : "batman" } + +> exit +bye +``` + +## Pause Database + +KubeDB takes advantage of `ValidationWebhook` feature in Kubernetes 1.9.0 or later clusters to implement `doNotPause` feature. If admission webhook is enabled, It prevents user from deleting the database as long as the `spec.doNotPause` is set to true. Since the MongoDB object created in this tutorial has `spec.doNotPause` set to true, if you delete the MongoDB object, KubeDB operator will nullify the delete operation. You can see this below: + +```console +$ kubedb delete mg mgo-quickstart -n demo +error: MongoDB "mgo-quickstart" can't be paused. To continue delete, unset spec.doNotPause and retry. +``` + +Now, run `kubedb edit mg mgo-quickstart -n demo` to set `spec.doNotPause` to false or remove this field (which default to false). Then if you delete the MongoDB object, KubeDB operator will delete the StatefulSet and its pods, but leaves the PVCs unchanged. In KubeDB parlance, we say that `mgo-quickstart` MongoDB database has entered into dormant state. This is represented by KubeDB operator by creating a matching DormantDatabase object. + +```yaml +$ kubedb delete mg mgo-quickstart -n demo +mongodb "mgo-quickstart" deleted + +$ kubedb get drmn -n demo mgo-quickstart +NAME STATUS AGE +mgo-quickstart Pausing 39s + +$ kubedb get drmn -n demo mgo-quickstart +NAME STATUS AGE +mgo-quickstart Paused 1m + + + +$ kubedb get drmn -n demo mgo-quickstart -o yaml +apiVersion: kubedb.com/v1alpha1 +kind: DormantDatabase +metadata: + clusterName: "" + creationTimestamp: 2018-02-02T09:24:49Z + finalizers: + - kubedb.com + generation: 0 + labels: + kubedb.com/kind: MongoDB + name: mgo-quickstart + namespace: demo + resourceVersion: "47107" + selfLink: /apis/kubedb.com/v1alpha1/namespaces/demo/dormantdatabases/mgo-quickstart + uid: eadf575b-07fa-11e8-946f-080027c05a6e +spec: + origin: + metadata: + creationTimestamp: null + name: mgo-quickstart + namespace: demo + spec: + mongodb: + databaseSecret: + secretName: mgo-quickstart-auth + resources: {} + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 50Mi + storageClassName: standard + version: "3.4" +status: + creationTime: 2018-02-02T09:24:50Z + pausingTime: 2018-02-02T09:25:11Z + phase: Paused +``` + +Here, + + - `spec.origin` is the spec of the original spec of the original MongoDB object. + + - `status.phase` points to the current database state `Paused`. + + +## Resume Dormant Database + +To resume the database from the dormant state, set `spec.resume` to `true` in the DormantDatabase object. + +```yaml +$ kubedb edit drmn -n demo mgo-quickstart +apiVersion: kubedb.com/v1alpha1 +kind: DormantDatabase +metadata: + name: mgo-quickstart + namespace: demo + ... +spec: + resume: true + ... +status: + phase: Paused + ... +``` + +KubeDB operator will notice that `spec.resume` is set to true. KubeDB operator will delete the DormantDatabase object and create a new MongoDB object using the original spec. This will in turn start a new StatefulSet which will mount the originally created PVCs. Thus the original database is resumed. + +Please note that the dormant database can also be resumed by creating same `MongoDB` database by using same Specs. In this tutorial, the dormant database can be resumed by creating `MongoDB` database using demo-1.yaml file. The below command resumes the dormant database `mgo-quickstart` that was created before. + +```console +$ kubedb create -f https://raw.githubusercontent.com/kubedb/cli/0.8.0-beta.1/docs/examples/mongodb/quickstart/demo-1.yaml +validating "https://raw.githubusercontent.com/kubedb/cli/0.8.0-beta.1/docs/examples/mongodb/quickstart/demo-1.yaml" +mongodb "mgo-quickstart" created +``` + +## Wipeout Dormant Database + +You can also wipe out a DormantDatabase by setting `spec.wipeOut` to true. KubeDB operator will delete the PVCs, delete any relevant Snapshot objects for this database and also delete snapshot data stored in the Cloud Storage buckets. There is no way to resume a wiped out database. So, be sure before you wipe out a database. + +```yaml +$ kubedb edit drmn -n demo mgo-quickstart +# set spec.wipeOut: true + +$ kubedb get drmn -n demo mgo-quickstart -o yaml +apiVersion: kubedb.com/v1alpha1 +kind: DormantDatabase +metadata: + clusterName: "" + creationTimestamp: 2018-02-02T09:30:06Z + finalizers: + - kubedb.com + generation: 0 + labels: + kubedb.com/kind: MongoDB + name: mgo-quickstart + namespace: demo + resourceVersion: "47440" + selfLink: /apis/kubedb.com/v1alpha1/namespaces/demo/dormantdatabases/mgo-quickstart + uid: a75e6314-07fb-11e8-946f-080027c05a6e +spec: + origin: + metadata: + creationTimestamp: null + name: mgo-quickstart + namespace: demo + spec: + mongodb: + databaseSecret: + secretName: mgo-quickstart-auth + resources: {} + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 50Mi + storageClassName: standard + version: "3.4" + wipeOut: true +status: + creationTime: 2018-02-02T09:30:07Z + pausingTime: 2018-02-02T09:30:21Z + phase: WipedOut + wipeOutTime: 2018-02-02T09:30:51Z + + +$ kubedb get drmn -n demo +NAME STATUS AGE +mgo-quickstart WipedOut 1m + +``` + +## Delete Dormant Database + +You still have a record that there used to be a MongoDB database `mgo-quickstart` in the form of a DormantDatabase database `mgo-quickstart`. Since you have already wiped out the database, you can delete the DormantDatabase object. + +```console +$ kubedb delete drmn mgo-quickstart -n demo +dormantdatabase "mgo-quickstart" deleted +``` + +## Cleaning up + +To cleanup the Kubernetes resources created by this tutorial, run: + +```console +$ kubedb delete mg mgo-quickstart -n demo --force +$ kubedb delete drmn mgo-quickstart -n demo --force + +# or +# $ kubedb delete mg,drmn,snap -n demo --all --force + +$ kubectl delete ns demo +namespace "demo" deleted +``` + + +## Next Steps +- [Snapshot and Restore](/docs/guides/mongodb/snapshot/backup-and-restore.md) process of MongoDB databases using KubeDB. +- Take [Scheduled Snapshot](/docs/guides/mongodb/snapshot/scheduled-backup.md) of MongoDB databases using KubeDB. +- Initialize [MongoDB with Script](/docs/guides/mongodb/initialization/using-script.md). +- Initialize [MongoDB with Snapshot](/docs/guides/mongodb/initialization/using-snapshot.md). +- Monitor your MongoDB database with KubeDB using [out-of-the-box CoreOS Prometheus Operator](/docs/guides/mongodb/monitoring/using-coreos-prometheus-operator.md). +- Monitor your MongoDB database with KubeDB using [out-of-the-box builtin-Prometheus](/docs/guides/mongodb/monitoring/using-builtin-prometheus.md). +- Use [Private Docker Registry](/docs/guides/mongodb/private-registry/using-private-registry.md) to deploy MongoDB with KubeDB. +- Detail concepts of [MongoDB object](/docs/concepts/databases/mongodb.md). +- Wondering what features are coming next? Please visit [here](/docs/roadmap.md). +- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). + diff --git a/docs/guides/mongodb/snapshot/backup-and-restore.md b/docs/guides/mongodb/snapshot/backup-and-restore.md new file mode 100644 index 000000000..cdf0e7ee9 --- /dev/null +++ b/docs/guides/mongodb/snapshot/backup-and-restore.md @@ -0,0 +1,304 @@ +> New to KubeDB? Please start [here](/docs/guides/README.md). + +# Database Snapshots +This tutorial will show you how to take snapshots of a KubeDB managed MongoDB database. + +## Before You Begin +At first, you need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one by using [Minikube](https://github.com/kubernetes/minikube). + +Now, install KubeDB cli on your workstation and KubeDB operator in your cluster following the steps [here](/docs/setup/install.md). + +A `MongoDB` database is needed to take snapshot for this tutorial. To keep things isolated, this tutorial uses a separate namespace called `demo` throughout this tutorial. Run the following command to prepare your cluster for this tutorial: + +```console +$ kubectl create -f https://raw.githubusercontent.com/kubedb/cli/0.8.0-beta.1/docs/examples/mongodb/demo-0.yaml +namespace "demo" created + +$ kubectl get ns +NAME STATUS AGE +default Active 1h +demo Active 1m +kube-public Active 1h +kube-system Active 1h + +$ kubedb create -f https://raw.githubusercontent.com/kubedb/cli/0.8.0-beta.1/docs/examples/mongodb/snapshot/demo-1.yaml +validating "https://raw.githubusercontent.com/kubedb/cli/0.8.0-beta.1/docs/examples/mongodb/snapshot/demo-1.yaml" +mongodb "mgo-infant" created +``` + +Please note that the yaml files that are used in this tutorial, stored in [docs/examples](https://github.com/kubedb/cli/tree/master/docs/examples) folder in GitHub repository [kubedb/cli](https://github.com/kubedb/cli). + +## Instant Backups +You can easily take a snapshot of `MongoDB` database by creating a `Snapshot` object. When a `Snapshot` object is created, KubeDB operator will launch a Job that runs the `mongodump` command and uploads the output bson file to various cloud providers S3, GCS, Azure, OpenStack Swift and/or locally mounted volumes using [osm](https://github.com/appscode/osm). + +In this tutorial, snapshots will be stored in a Google Cloud Storage (GCS) bucket. To do so, a secret is needed that has the following 2 keys: + +| Key | Description | +|-----------------------------------|------------------------------------------------------------| +| `GOOGLE_PROJECT_ID` | `Required`. Google Cloud project ID | +| `GOOGLE_SERVICE_ACCOUNT_JSON_KEY` | `Required`. Google Cloud service account json key | + +```console +$ echo -n '' > GOOGLE_PROJECT_ID +$ mv downloaded-sa-json.key > GOOGLE_SERVICE_ACCOUNT_JSON_KEY +$ kubectl create secret generic mg-snap-secret -n demo \ + --from-file=./GOOGLE_PROJECT_ID \ + --from-file=./GOOGLE_SERVICE_ACCOUNT_JSON_KEY +secret "mg-snap-secret" created +``` + +```yaml +$ kubectl get secret mg-snap-secret -n demo -o yaml +apiVersion: v1 +data: + GOOGLE_PROJECT_ID: PHlvdXItcHJvamVjdC1pZD4= + GOOGLE_SERVICE_ACCOUNT_JSON_KEY: ewogICJ0eXBlIjogInNlcnZpY2VfYWNjb3V...9tIgp9Cg== +kind: Secret +metadata: + creationTimestamp: 2018-02-02T10:02:09Z + name: mg-snap-secret + namespace: demo + resourceVersion: "48679" + selfLink: /api/v1/namespaces/demo/secrets/mg-snap-secret + uid: 220a7c60-0800-11e8-946f-080027c05a6e +type: Opaque +``` + +To lean how to configure other storage destinations for Snapshots, please visit [here](/docs/concepts/snapshot.md). Now, create the Snapshot object. + +```yaml +apiVersion: kubedb.com/v1alpha1 +kind: Snapshot +metadata: + name: snapshot-infant + namespace: demo + labels: + kubedb.com/kind: MongoDB +spec: + databaseName: mgo-infant + storageSecretName: mg-snap-secret + gcs: + bucket: restic +``` + +```console +$ kubedb create -f https://raw.githubusercontent.com/kubedb/cli/0.8.0-beta.1/docs/examples/mongodb/snapshot/demo-2.yaml +validating "https://raw.githubusercontent.com/kubedb/cli/0.8.0-beta.1/docs/examples/mongodb/snapshot/demo-2.yaml" +snapshot "snapshot-infant" created + +$ kubedb get snap -n demo +NAME DATABASE STATUS AGE +snapshot-infant mg/mgo-infant Running 47s +``` + +```yaml +$ kubedb get snap -n demo snapshot-infant -o yaml +apiVersion: kubedb.com/v1alpha1 +kind: Snapshot +metadata: + clusterName: "" + creationTimestamp: 2018-02-02T10:05:36Z + finalizers: + - kubedb.com + generation: 0 + labels: + kubedb.com/kind: MongoDB + kubedb.com/name: mgo-infant + name: snapshot-infant + namespace: demo + resourceVersion: "48991" + selfLink: /apis/kubedb.com/v1alpha1/namespaces/demo/snapshots/snapshot-infant + uid: 9d4f37a0-0800-11e8-946f-080027c05a6e +spec: + databaseName: mgo-infant + gcs: + bucket: restic + storageSecretName: mg-snap-secret +status: + completionTime: 2018-02-02T10:06:43Z + phase: Succeeded + startTime: 2018-02-02T10:05:37Z +``` + +Here, + +- `metadata.labels` should include the type of database `kubedb.com/kind: MongoDB` whose snapshot will be taken. + +- `spec.databaseName` points to the database whose snapshot is taken. + +- `spec.storageSecretName` points to the Secret containing the credentials for snapshot storage destination. + +- `spec.gcs.bucket` points to the bucket name used to store the snapshot data. + + +You can also run the `kubedb describe` command to see the recent snapshots taken for a database. + +```console +$ kubedb describe mg -n demo mgo-infant +Name: mgo-infant +Namespace: demo +StartTimestamp: Fri, 02 Feb 2018 16:04:50 +0600 +Status: Running +Volume: + StorageClass: standard + Capacity: 50Mi + Access Modes: RWO + +StatefulSet: + Name: mgo-infant + Replicas: 1 current / 1 desired + CreationTimestamp: Fri, 02 Feb 2018 16:04:56 +0600 + Pods Status: 1 Running / 0 Waiting / 0 Succeeded / 0 Failed + +Service: + Name: mgo-infant + Type: ClusterIP + IP: 10.99.34.23 + Port: db 27017/TCP + +Database Secret: + Name: mgo-infant-auth + Type: Opaque + Data + ==== + password: 16 bytes + user: 4 bytes + +Snapshots: + Name Bucket StartTime CompletionTime Phase + ---- ------ --------- -------------- ----- + snapshot-infant gs:restic Fri, 02 Feb 2018 16:05:37 +0600 Fri, 02 Feb 2018 16:06:43 +0600 Succeeded + +Events: + FirstSeen LastSeen Count From Type Reason Message + --------- -------- ----- ---- -------- ------ ------- + 8m 8m 1 Job Controller Normal SuccessfulSnapshot Successfully completed snapshot + 9m 9m 1 Snapshot Controller Normal Starting Backup running + 9m 9m 1 MongoDB operator Normal Successful Successfully patched StatefulSet + 9m 9m 1 MongoDB operator Normal Successful Successfully patched MongoDB + 9m 9m 1 MongoDB operator Normal Successful Successfully created StatefulSet + 9m 9m 1 MongoDB operator Normal Successful Successfully created MongoDB + 10m 10m 1 MongoDB operator Normal Successful Successfully created Service +``` + + +Once the snapshot Job is complete, you should see the output of the `mongodump` command stored in the GCS bucket. + +![snapshot-console](/docs/images/mongodb/mgo-xyz-snapshot.png) + +From the above image, you can see that the snapshot output is stored in a folder called `{bucket}/kubedb/{namespace}/{mongodb-object}/{snapshot}/`. + + +## Restore from Snapshot +You can create a new database from a previously taken Snapshot. Specify the Snapshot name in the `spec.init.snapshotSource` field of a new MongoDB object. See the example `mgo-recovered` object below: + +```yaml +apiVersion: kubedb.com/v1alpha1 +kind: MongoDB +metadata: + name: mgo-recovered + namespace: demo +spec: + version: 3.4 + storage: + storageClassName: "standard" + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 50Mi + init: + snapshotSource: + name: snapshot-infant + namespace: demo +``` + +```console +$ kubedb create -f https://raw.githubusercontent.com/kubedb/cli/0.8.0-beta.1/docs/examples/mongodb/snapshot/demo-3.yaml +validating "https://raw.githubusercontent.com/kubedb/cli/0.8.0-beta.1/docs/examples/mongodb/snapshot/demo-3.yaml" +mongodb "mgo-recovered" created +``` + +Here, + + - `spec.init.snapshotSource.name` refers to a Snapshot object for a MongoDB database in the same namespaces as this new `mgo-recovered` MongoDB object. + +Now, wait several seconds. KubeDB operator will create a new StatefulSet. Then KubeDB operator launches a Kubernetes Job to initialize the new database using the data from `mgo-xyz` Snapshot. + +```console +$ kubedb get mg -n demo +NAME STATUS AGE +mgo-infant Running 23m +mgo-recovered Running 4m + +$ kubedb describe mg -n demo mgo-recovered +Name: mgo-recovered +Namespace: demo +StartTimestamp: Fri, 02 Feb 2018 16:24:23 +0600 +Status: Running +Annotations: kubedb.com/initialized= +Volume: + StorageClass: standard + Capacity: 50Mi + Access Modes: RWO + +StatefulSet: + Name: mgo-recovered + Replicas: 1 current / 1 desired + CreationTimestamp: Fri, 02 Feb 2018 16:24:36 +0600 + Pods Status: 1 Running / 0 Waiting / 0 Succeeded / 0 Failed + +Service: + Name: mgo-recovered + Type: ClusterIP + IP: 10.107.157.253 + Port: db 27017/TCP + +Database Secret: + Name: mgo-recovered-auth + Type: Opaque + Data + ==== + user: 4 bytes + password: 16 bytes + +No Snapshots. + +Events: + FirstSeen LastSeen Count From Type Reason Message + --------- -------- ----- ---- -------- ------ ------- + 3m 3m 1 MongoDB operator Normal Successful Successfully patched StatefulSet + 3m 3m 1 MongoDB operator Normal Successful Successfully patched MongoDB + 3m 3m 1 Job Controller Normal SuccessfulInitialize Successfully completed initialization + 4m 4m 1 MongoDB operator Normal Successful Successfully patched StatefulSet + 4m 4m 1 MongoDB operator Normal Successful Successfully patched MongoDB + 4m 4m 1 MongoDB operator Normal Initializing Initializing from Snapshot: "snapshot-infant" + 4m 4m 1 MongoDB operator Normal Successful Successfully created StatefulSet + 4m 4m 1 MongoDB operator Normal Successful Successfully created MongoDB + 4m 4m 1 MongoDB operator Normal Successful Successfully created Service +``` + + +## Cleaning up +To cleanup the Kubernetes resources created by this tutorial, run: + +```console +$ kubedb delete mg,drmn,snap -n demo --all --force + +$ kubectl delete ns demo +namespace "demo" deleted +``` + + +## Next Steps +- See the list of supported storage providers for snapshots [here](/docs/concepts/snapshot.md). +- Take [Scheduled Snapshot](/docs/guides/mongodb/snapshot/scheduled-backup.md) of MongoDB databases using KubeDB. +- Initialize [MongoDB with Script](/docs/guides/mongodb/initialization/using-script.md). +- Initialize [MongoDB with Snapshot](/docs/guides/mongodb/initialization/using-snapshot.md). +- Monitor your MongoDB database with KubeDB using [out-of-the-box CoreOS Prometheus Operator](/docs/guides/mongodb/monitoring/using-coreos-prometheus-operator.md). +- Monitor your MongoDB database with KubeDB using [out-of-the-box builtin-Prometheus](/docs/guides/mongodb/monitoring/using-builtin-prometheus.md). +- Use [Private Docker Registry](/docs/guides/mongodb/private-registry/using-private-registry.md) to deploy MongoDB with KubeDB. +- Detail concepts of [MongoDB object](/docs/concepts/databases/mongodb.md). +- Wondering what features are coming next? Please visit [here](/docs/roadmap.md). +- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). + diff --git a/docs/guides/mongodb/snapshot/scheduled-backup.md b/docs/guides/mongodb/snapshot/scheduled-backup.md new file mode 100644 index 000000000..646947c7a --- /dev/null +++ b/docs/guides/mongodb/snapshot/scheduled-backup.md @@ -0,0 +1,190 @@ +> New to KubeDB? Please start [here](/docs/guides/README.md). + +# Database Scheduled Snapshots +This tutorial will show you how to use KubeDB to take scheduled snapshot of a MongoDB database. + +## Before You Begin +At first, you need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one by using [Minikube](https://github.com/kubernetes/minikube). + +Now, install KubeDB cli on your workstation and KubeDB operator in your cluster following the steps [here](/docs/setup/install.md). + +To keep things isolated, this tutorial uses a separate namespace called `demo` throughout this tutorial. Run the following command to prepare your cluster for this tutorial: + +```console +$ kubectl create -f https://raw.githubusercontent.com/kubedb/cli/0.8.0-beta.1/docs/examples/mongodb/demo-0.yaml +namespace "demo" created + +$ kubectl get ns +NAME STATUS AGE +default Active 1h +demo Active 1m +kube-public Active 1h +kube-system Active 1h +``` + +Please note that the yaml files that are used in this tutorial, stored in [docs/examples](https://github.com/kubedb/cli/tree/master/docs/examples) folder in GitHub repository [kubedb/cli](https://github.com/kubedb/cli). + + +## Scheduled Backups +KubeDB supports taking periodic backups for a database using a [cron expression](https://github.com/robfig/cron/blob/v2/doc.go#L26). KubeDB operator will launch a Job periodically that runs the `mongodump` command and uploads the output bson file to various cloud providers S3, GCS, Azure, OpenStack Swift and/or locally mounted volumes using [osm](https://github.com/appscode/osm). + +In this tutorial, snapshots will be stored in a Google Cloud Storage (GCS) bucket. To do so, a secret is needed that has the following 2 keys: + +| Key | Description | +|-----------------------------------|------------------------------------------------------------| +| `GOOGLE_PROJECT_ID` | `Required`. Google Cloud project ID | +| `GOOGLE_SERVICE_ACCOUNT_JSON_KEY` | `Required`. Google Cloud service account json key | + +```console +$ echo -n '' > GOOGLE_PROJECT_ID +$ mv downloaded-sa-json.key > GOOGLE_SERVICE_ACCOUNT_JSON_KEY +$ kubectl create secret generic mg-snap-secret -n demo \ + --from-file=./GOOGLE_PROJECT_ID \ + --from-file=./GOOGLE_SERVICE_ACCOUNT_JSON_KEY +secret "mg-snap-secret" created +``` + + +```yaml +$ kubectl get secret mg-snap-secret -n demo -o yaml +apiVersion: v1 +data: + GOOGLE_PROJECT_ID: PHlvdXItcHJvamVjdC1pZD4= + GOOGLE_SERVICE_ACCOUNT_JSON_KEY: ewogICJ0eXBlIjogInNlcnZpY2VfYWNjb3V...9tIgp9Cg== +kind: Secret +metadata: + creationTimestamp: 2018-02-02T10:02:09Z + name: mg-snap-secret + namespace: demo + resourceVersion: "48679" + selfLink: /api/v1/namespaces/demo/secrets/mg-snap-secret + uid: 220a7c60-0800-11e8-946f-080027c05a6e +type: Opaque +``` + + +To learn how to configure other storage destinations for Snapshots, please visit [here](/docs/concepts/snapshot.md). Now, create the `MongoDB` object with scheduled snapshot. + +```yaml +apiVersion: kubedb.com/v1alpha1 +kind: MongoDB +metadata: + name: mgo-scheduled + namespace: demo +spec: + version: 3.4 + storage: + storageClassName: "standard" + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 50Mi + init: + scriptSource: + gitRepo: + repository: "https://github.com/kubedb/mongodb-init-scripts.git" + directory: . + backupSchedule: + cronExpression: "@every 1m" + storageSecretName: mg-snap-secret + gcs: + bucket: restic +``` + + +```console +$ kubedb create -f https://raw.githubusercontent.com/kubedb/cli/0.8.0-beta.1/docs/examples/mongodb/snapshot/demo-4.yaml +validating "https://raw.githubusercontent.com/kubedb/cli/0.8.0-beta.1/docs/examples/mongodb/snapshot/demo-4.yaml" +mongodb "mgo-scheduled" created +``` + +It is also possible to add backup scheduler to an existing `MongoDB`. You just have to edit the `MongoDB` CRD and add below spec: +```yaml +$ kubedb edit mg {db-name} -n demo +spec: + backupSchedule: + cronExpression: '@every 1m' + gcs: + bucket: restic + storageSecretName: mg-snap-secret +``` + +Once the `spec.backupSchedule` is added, KubeDB operator will create a new Snapshot object on each tick of the cron expression. This triggers KubeDB operator to create a Job as it would for any regular instant backup process. You can see the snapshots as they are created using `kubedb get snap` command. + +```console +$ kubedb get snap -n demo +NAME DATABASE STATUS AGE +mgo-scheduled-20180202-104632 mg/mgo-scheduled Succeeded 4m +mgo-scheduled-20180202-104737 mg/mgo-scheduled Succeeded 3m +mgo-scheduled-20180202-104837 mg/mgo-scheduled Succeeded 2m +mgo-scheduled-20180202-104937 mg/mgo-scheduled Running 10s +``` + +you should see the output of the `mongodump` command for each snapshot stored in the GCS bucket. + +![snapshot-console](/docs/images/mongodb/mgo-scheduled.png) + +From the above image, you can see that the snapshot output is stored in a folder called `{bucket}/kubedb/{namespace}/{mongodb-object}/{snapshot}/`. + + +## Remove Scheduler +To remove scheduler, edit the MongoDB object to remove `spec.backupSchedule` section. + +```yaml +$ kubedb edit mg mgo-scheduled -n demo +apiVersion: kubedb.com/v1alpha1 +kind: MongoDB +metadata: + name: mgo-scheduled + namespace: demo + ... +spec: +# backupSchedule: +# cronExpression: '@every 1m' +# gcs: +# bucket: restic +# storageSecretName: mg-snap-secret + databaseSecret: + secretName: mgo-scheduled-auth + init: + scriptSource: + gitRepo: + directory: . + repository: https://github.com/kubedb/mongodb-init-scripts.git + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 50Mi + storageClassName: standard + version: 3.4 +status: + creationTime: 2018-02-02T10:46:18Z + phase: Running + +``` + +## Cleaning up +To cleanup the Kubernetes resources created by this tutorial, run: + +```console +$ kubedb delete mg,drmn,snap -n demo --all --force + +$ kubectl delete ns demo +namespace "demo" deleted +``` + + +## Next Steps +- See the list of supported storage providers for snapshots [here](/docs/concepts/snapshot.md). +- Initialize [MongoDB with Script](/docs/guides/mongodb/initialization/using-script.md). +- Initialize [MongoDB with Snapshot](/docs/guides/mongodb/initialization/using-snapshot.md). +- Monitor your MongoDB database with KubeDB using [out-of-the-box CoreOS Prometheus Operator](/docs/guides/mongodb/monitoring/using-coreos-prometheus-operator.md). +- Monitor your MongoDB database with KubeDB using [out-of-the-box builtin-Prometheus](/docs/guides/mongodb/monitoring/using-builtin-prometheus.md). +- Use [Private Docker Registry](/docs/guides/mongodb/private-registry/using-private-registry.md) to deploy MongoDB with KubeDB. +- Detail concepts of [MongoDB object](/docs/concepts/databases/mongodb.md). +- Wondering what features are coming next? Please visit [here](/docs/roadmap.md). +- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). + diff --git a/docs/images/mongodb/builtin-prometheus.png b/docs/images/mongodb/builtin-prometheus.png new file mode 100644 index 000000000..26b521eac Binary files /dev/null and b/docs/images/mongodb/builtin-prometheus.png differ diff --git a/docs/images/mongodb/mgo-lifecycle.png b/docs/images/mongodb/mgo-lifecycle.png new file mode 100644 index 000000000..7bfad6fca Binary files /dev/null and b/docs/images/mongodb/mgo-lifecycle.png differ diff --git a/docs/images/mongodb/mgo-scheduled.png b/docs/images/mongodb/mgo-scheduled.png new file mode 100644 index 000000000..f7e3380d8 Binary files /dev/null and b/docs/images/mongodb/mgo-scheduled.png differ