Merge pull request #134 from kanisterio/sync

Update docs; Add EncryptionKey arg to kanister fucnction; Revert objectstore helper func

docs/architecture.rst

@@ -73,8 +73,10 @@ The definition of a `BlueprintAction` is:
       Phases             []BlueprintPhase    `json:"phases"`
   }
 
-- `Kind` is required as and represents the type of object to act on.
-   Currently the supported values are `Deployment` or `Statefulset`.
+- `Kind` represents the type of Kubernetes object this BlueprintAction is written for.
+  Specifying this is optional and going forward, if this is specified, Kanister will
+  enforce that it matches the `Object` kind specified in an ActionSet referencing this
+  BlueprintAction
 - `ConfigMapNames`, `SecretNames`, `InputArtifactNames` are optional
   but, if specified, they list named parameters that must be included by
   the `ActionSet`.

docs/functions.rst

@@ -257,6 +257,8 @@ Example:
         - |
           cp /restore-data/file_to_replace.data /data/file.data
 
+.. _backupdata:
+
 BackupData
 ----------
 
@@ -294,6 +296,8 @@ Example:
       backupArtifactPrefix: s3-bucket/path/artifactPrefix
       backupIdentifier: "{{ .Time }}"
 
+.. _restoredata:
+
 RestoreData
 -----------
 
@@ -351,6 +355,54 @@ Example:
       backupArtifactPrefix: s3-bucket/path/artifactPrefix
       backupIdentifier: "{{ .Time }}"
 
+CopyVolumeData
+--------------
+
+This function copies data from the specified volume (referenced by a
+Kubernetes PersistentVolumeClaim) into an object store.
+This data can be restored into a volume using the :ref:`restoredata`
+function
+
+.. note::
+   The PVC must not be in-use (attached to a running Pod)
+
+   If data needs to be copied from a running workload without stopping
+   it, use the :ref:`backupdata` function
+
+Arguments:
+
+.. csv-table::
+   :header: "Argument", "Required", "Type", "Description"
+   :align: left
+   :widths: 5,5,5,15
+
+   `namespace`, Yes, `string`, namespace the source PVC is in
+   `volume`, Yes, `string`, name of the source PVC
+   `dataArtifactPrefix`, Yes, `string`, path on the object store to store the data in
+
+Outputs:
+
+.. csv-table::
+   :header: "Output", "Type", "Description"
+   :align: left
+   :widths: 5,5,15
+
+   `backupArtifactLocation`,`string`, location in objectstore where data was copied
+   `backupID`,`string`,  unique string to identify this data copy
+
+Example:
+
+If the ActionSet `Object` is a PersistentVolumeClaim:
+
+.. code-block:: yaml
+  :linenos:
+
+  - func: CopyVolumeData
+    args:
+      namespace: "{{ .PVC.Namespace }}"
+      volume: "{{ .PVC.Name }}"
+      dataArtifactPrefix: s3-bucket-name/path
+
 DeleteData
 ----------
 

docs/helm.rst

@@ -39,3 +39,4 @@ The following application-specific instructions are available:
    helm_instructions/mysql_instructions.rst
    helm_instructions/pgsql_instructions.rst
    helm_instructions/mongodb_instructions.rst
+   helm_instructions/elasticsearch_instructions.rst

docs/helm_instructions/create_profile.rst

@@ -11,7 +11,7 @@ create as part of the first instance.
 
    * Create a bucket for artifacts on your S3 store. This will be your
      ``s3.bucket`` parameter to the command.
-   * Obtain ``s3.apiKey`` and ``s3.secretKey`` credentials for an
+   * Obtain ``s3.accessKey`` and ``s3.secretKey`` credentials for an
      account with access to the bucket that you will use.
    * Configure the permissions on the bucket to allow the account to
      list, put, get, and delete.

docs/helm_instructions/elasticsearch_instructions.rst

@@ -0,0 +1,59 @@
+Kanister-Enabled Elasticsearch
+------------------------------
+
+This section describes the steps for a basic installation of an instance of
+Elasticsearch (a stateful set application running master, client and data
+nodes with persistent volumes attached to master and data nodes) along with
+a Kanister Blueprint and a Profile via a Kanister-enabled Helm chart.
+
+.. code-block:: console
+
+   $ helm repo add kanister https://charts.kanister.io/
+
+
+Then install the sample Elasticsearch application in its own namespace.
+
+.. For some reason using 'console' or 'bash' highlights the snippet weirdly
+
+.. only:: kanister
+
+  .. code-block:: rst
+
+     # Install Kanister-enabled Elasticsearch
+     $ helm install kanister/kanister-elasticsearch -n elasticsearch \
+          --namespace es-test \
+          --set profile.create='true' \
+          --set profile.profileName='es-test-profile' \
+          --set profile.s3.endpoint='https://my-custom-s3-provider:9000' \
+          --set profile.s3.accessKey='AKIAIOSFODNN7EXAMPLE' \
+          --set profile.s3.secretKey='wJalrXUtnFEMI%K7MDENG%bPxRfiCYEXAMPLEKEY' \
+          --set profile.s3.bucket='kanister-bucket'
+          --set kanister.controller_namespace="kanister"
+
+.. only:: defaultns
+
+  .. code-block:: rst
+
+     # Install Kanister-enabled Elasticsearch
+     $ helm install kanister/kanister-elasticsearch -n elasticsearch --namespace es-test
+
+
+The settings in the command above represent the minimum recommended set for
+your installation.
+
+.. only:: kanister
+
+  .. include:: ./create_profile.rst
+
+  If not creating a Profile CR, it is possible to use an even simpler command.
+
+  .. code-block:: rst
+
+     # Install Kanister-enabled Elasticsearch
+     $ helm install kanister/kanister-elasticsearch -n elasticsearch --namespace es-test
+
+.. note:: The above command will attempt to use dynamic storage provisioning
+   based on the the default storage class for your cluster. You will to need to
+   `designate a default storage class <https://kubernetes.io/docs/tasks/administer-cluster/change-default-storage-class/#changing-the-default-storageclass>`_
+   or, use a specific storage class by providing a value with the
+   ``--set master.persistence.storageClass`` and ``--set data.persistence.storageClass`` option.

docs/helm_instructions/mongodb_instructions.rst

@@ -25,10 +25,10 @@ Then install the sample MongoDB replica set application in its own namespace.
           --set profile.profileName='mongo-test-profile' \
           --set profile.s3.bucket="kanister-bucket" \
           --set profile.s3.endpoint="https://my-custom-s3-provider:9000" \
-          --set profile.s3.apiKey="AKIAIOSFODNN7EXAMPLE" \
+          --set profile.s3.accessKey="AKIAIOSFODNN7EXAMPLE" \
           --set profile.s3.secretKey="wJalrXUtnFEMI!K7MDENG!bPxRfiCYEXAMPLEKEY" \
           --set kanister.controller_namespace="kanister" \
-          --set resplicas=1 \
+          --set replicas=1 \
           --set persistentVolume.size=2Gi
 
 .. only:: defaultns

docs/helm_instructions/mysql_instructions.rst

@@ -24,7 +24,7 @@ Then install the sample MySQL application in its own namespace.
           --set profile.profileName='mysql-test-profile' \
           --set profile.s3.bucket="kanister-bucket" \
           --set profile.s3.endpoint="https://my-custom-s3-provider:9000" \
-          --set profile.s3.apiKey="AKIAIOSFODNN7EXAMPLE" \
+          --set profile.s3.accessKey="AKIAIOSFODNN7EXAMPLE" \
           --set profile.s3.secretKey="wJalrXUtnFEMI!K7MDENG!bPxRfiCYEXAMPLEKEY" \
           --set kanister.controller_namespace="kanister" \
           --set mysqlRootPassword="asd#45@mysqlEXAMPLE" \

docs/helm_instructions/pgsql_instructions.rst

@@ -28,7 +28,7 @@ Then install the sample PostgreSQL application in its own namespace.
           --set profile.profileName='postgres-test-profile' \
           --set profile.s3.bucket="kanister-bucket" \
           --set profile.s3.endpoint="https://my-custom-s3-provider:9000" \
-          --set profile.s3.apiKey="AKIAIOSFODNN7EXAMPLE" \
+          --set profile.s3.accessKey="AKIAIOSFODNN7EXAMPLE" \
           --set profile.s3.secretKey="wJalrXUtnFEMI!K7MDENG!bPxRfiCYEXAMPLEKEY" \
           --set kanister.controller_namespace="kanister"
 

docs/spelling_wordlist.txt

@@ -25,3 +25,7 @@ defaultProfile
 Kando
 kando
 Dockerfile
+PersistentVolumeClaim
+actionset
+objectstore
+Elasticsearch

docs/templates.rst

@@ -15,12 +15,14 @@ The TemplateParam struct is defined as:
       StatefulSet  StatefulSetParams
       Deployment   DeploymentParams
       PVC          PVCParams
+      Namespace    NamespaceParams
       ArtifactsIn  map[string]crv1alpha1.Artifact // A Kanister Artifact
       Profile      *Profile
       ConfigMaps   map[string]v1.ConfigMap
       Secrets      map[string]v1.Secret
       Time         string
       Options      map[string]string
+      Object       map[string]interface{}
       Phases       map[string]*Phase
   }
 
@@ -51,16 +53,20 @@ standard go template functions, Kanister imports all the `sprig
     return ras, nil
 
 
-Protected Objects
-=================
+Objects
+=======
 
-Kanister operates on the granularity of a `ProtectedObject`. As of the current
-release, a Protected Object is a workload, specifically, a Deployment or
-StatefulSet. The TemplateParams struct has one field for each potential protected
-object, which is effectively a union in go.
+Kanister operates on the granularity of an `Object`. As of the current
+release, well known Object types are `Deployment`, `StatefulSet`, `PersistentVolumeClaim`,
+or `Namespace`. The TemplateParams struct has one field for each well known
+object type, which is effectively a union in go.
 
-Each ProtectedObject param struct is a set of useful fields related to the
-ProtectedObject.
+Other than the types mentioned above, Kanister can also act on any Kubernetes
+object such as a CRD and the :ref:`object` field in TemplateParams is populated with the
+unstructured content of those.
+
+Each param struct described below is a set of useful fields related to the
+Object.
 
 StatefulSet
 -----------
@@ -109,6 +115,27 @@ For example, to access the Name of a Deployment use:
 
   "{{ index .Deployment.Name }}"
 
+Namespace
+---
+
+NamespaceParams includes the name of the namespace
+that is being acted on when the ActionSet `Object` is
+specifies a Namespace
+
+.. code-block:: go
+  :linenos:
+
+  // NamespaceParams are params for a Namespace
+  type NamespaceParams struct {
+    Name              string
+  }
+
+For example, to access the Name of a Namespace, use:
+
+.. code-block:: go
+
+  "{{ .Namespace.Name }}"
+
 PVC
 ---
 
@@ -130,6 +157,32 @@ For example, to access the Name of a persistent volume claim, use:
 
   "{{ .PVC.Name }}"
 
+.. _object:
+
+Object
+------
+
+Object includes the unstructured representation of the underlying
+Kubernetes object. This allows the flexibility of writing blueprints
+that operate on objects that are not well known to Kanister such as
+CRD's
+
+.. code-block:: go
+  :linenos:
+
+  type TemplateParams struct {
+    ...
+    Object       map[string]interface{}
+    ...
+  }
+
+For example, to access the Name in the Kubernetes ObjectMeta of an
+arbitrary object, use:
+
+.. code-block:: go
+
+  "{{ .Object.metadata.name }}"
+
 Artifacts
 =========
 

docs/tooling.rst

@@ -71,6 +71,8 @@ As seen above, both ActionSets and profiles can be created using `kanctl create`
     -f, --from string                 specify name of the action set
     -h, --help                        help for actionset
     -k, --kind string                 resource kind to apply selector on. Used along with the selector specified using --selector/-l (default "all")
+    -T, --namespacetargets strings    namespaces for the action set, comma separated list of namespaces (eg: --namespacetargets namespace1,namespace2)
+    -O, --objects strings             objects for the action set, comma separated list of object references (eg: --objects group/version/resource/namespace1/name1,group/version/resource/namespace2/name2)
     -o, --options strings             specify options for the action set, comma separated key=value pairs (eg: --options key1=value1,key2=value2)
     -p, --profile string              profile for the action set
     -v, --pvc strings                 pvc for the action set, comma separated namespace/name pairs (eg: --pvc namespace1/name1,namespace2/name2)
@@ -295,14 +297,18 @@ object store like AWS S3. We've found it can be cumbersome to pass Profile
 configuration to tools like the AWS command line from inside Blueprints.
 
 `kando` is a tool to simplify object store interactions from within blueprints.
-It has three commands:
+It also provides a way to create desired output from a blueprint phase.
+
+It has the following commands:
 
 * `location push`
 
 * `location pull`
 
 * `location delete`
 
+* `output`
+
 The usage for these commands can be displayed using the `--help` flag:
 
 .. code-block:: bash
@@ -337,18 +343,29 @@ The usage for these commands can be displayed using the `--help` flag:
 
 .. code-block:: bash
 
-$ kando location delete --help
-Delete artifacts from s3-compliant object storage
+  $ kando location delete --help
+  Delete artifacts from s3-compliant object storage
+
+  Usage:
+    kando location delete [flags]
+
+  Flags:
+    -h, --help   help for delete
+
+  Global Flags:
+    -s, --path string      Specify a path suffix (optional)
+    -p, --profile string   Pass a Profile as a JSON string (required)
 
-Usage:
-  kando location delete [flags]
+.. code-block:: bash
 
-Flags:
-  -h, --help   help for delete
+  $ kando output --help
+  Create phase output with given key:value
 
-Global Flags:
-  -s, --path string      Specify a path suffix (optional)
-  -p, --profile string   Pass a Profile as a JSON string (required)
+  Usage:
+    kando output <key> <value> [flags]
+
+  Flags:
+    -h, --help   help for output
 
 The following snippet is an example of using kando from inside a Blueprint.
 
@@ -358,6 +375,7 @@ The following snippet is an example of using kando from inside a Blueprint.
 
   kando location delete --profile '{{ .Profile }}' --path '/backup/path'
 
+  kando output version 0.13.0
 
 Docker Image
 ============