RepositoryServer support in kando command line #1951

r4rajat · 2023-03-12T11:13:17Z

Change Overview

Support for passing --repository-server argument in kando location commands.

Pull request type

Please check the type of change your PR introduces:

Issues

fixes #issue-number

Test Plan

💪 Manual
⚡ Unit test
💚 E2E

Manuial Testing

1) Create a Cluster with the Repository Server Controller Deployment and Repository Server Actionset support.

--namespace kanister \
--set image.repository=r4rajat/controller \
--set image.tag=v3-test-rajat \
--set repositoryServerControllerImage.registry=r4rajat \
--set repositoryServerControllerImage.name=repo-server-controller \
--set repositoryServerControllerImage.tag=v3-test-rajat \
--create-namespace

2) Apply Repo Server CRD

kubectl apply -f pkg/customresource/repositoryserver.yaml -n kanister

3) Create Test Application [MySQL]

helm install mysql-release bitnami/mysql --namespace mysql --set auth.rootPassword='asd#45@mysqlEXAMPLE' --create-namespace

4) Create OpenSSL Certificate

openssl req -newkey rsa:2048 -nodes -keyout key.pem -x509 -days 365 -out certificate.pem

5) Create S3 Location and Location Secret Config Files

S3 Location Secret

vi s3_location_creds.yaml

apiVersion: v1
kind: Secret
metadata:
   name: s3-creds
   namespace: kanister
   labels:
      repo.kanister.io/target-namespace: monitoring
type: secrets.kanister.io/aws
data:
   # required: base64 encoded value for key with proper permissions for the bucket
   aws_access_key_id: <base64 encoded access key>
   # required: base64 encoded value for the secret corresponding to the key above
   aws_secret_access_key: <base64 encoded secret key>

S3 Location

vi s3_location.yaml

apiVersion: v1
kind: Secret
metadata:
   name: s3-location
   namespace: kanister
   labels:
      repo.kanister.io/target-namespace: monitoring
type: Opaque
data:
   # required: specify the type of the store
   # supported values are s3, gcs, azure, and file-store
   type: czM=
   bucket: cmFqYXQtaW5mcmFjbG91ZA==
   # optional: used as a sub path in the bucket for all backups
   path: L3JlcG8tY29udHJvbGxlci8=
   # required, if supported by the provider
   region: dXMtZWFzdC0x
   # optional: if set to true, do not verify SSL cert.
   # Default, when omitted, is false
   #skipSSLVerify: false
   # required: if type is `file-store`
   # optional, otherwise
   #claimName: store-pvc

6) Apply Secrets

kubectl create secret tls repository-server-tls-cert --cert=certificate.pem --key=key.pem -n kanister

kubectl create secret generic repository-server-user-access -n kanister --from-literal=localhost=test1234

kubectl create secret generic repository-admin-user -n kanister --from-literal=username=admin@testpod1 --from-literal=password=test1234

kubectl create secret generic repo-pass -n kanister --from-literal=repo-password=test1234

kubectl apply -f s3_location_creds.yaml -n kanister

kubectl apply -f s3_location.yaml -n kanister

7) Create Repository

kopia --log-level=error --config-file=/tmp/kopia-repository.config --log-dir=/tmp/kopia-cache repository create --no-check-for-updates --cache-directory=/tmp/cache.dir --content-cache-size-mb=0 --metadata-cache-size-mb=500 --override-hostname=mysql.app --override-username=kanisterAdmin s3 --bucket=rajat-infracloud --prefix=/repo-controller/ --region=us-east-1 --access-key=<ACCESS_KEY> --secret-access-key=<SECRET_ACCESS_KEY>

8) Create Repository Server CR

vi repo-server-cr.yaml

apiVersion: cr.kanister.io/v1alpha1
kind: RepositoryServer
metadata:
  labels:
    app.kubernetes.io/name: repositoryserver
    app.kubernetes.io/instance: repositoryserver-sample
    app.kubernetes.io/part-of: kanister
    app.kuberentes.io/managed-by: kustomize
    app.kubernetes.io/created-by: kanister
  name: kopia-repo-server-1
  namespace: kanister
spec:
  storage:
    secretRef:
      name: s3-location
      namespace: kanister
    credentialSecretRef:
      name: s3-creds
      namespace: kanister
  repository:
    rootPath: /repo-controller/
    passwordSecretRef:
      name: repo-pass
      namespace: kanister
    username: kanisterAdmin
    hostname: mysql.app
  server:
    adminSecretRef:
      name: repository-admin-user
      namespace: kanister
    tlsSecretRef:
      name: repository-server-tls-cert
      namespace: kanister
    userAccess:
      userAccessSecretRef:
        name: repository-server-user-access
        namespace: kanister
      username: kanisteruser

kubectl apply -f repo-server-cr.yaml -n kanister

Wait till the status of Repository Server CR gets to ServerReady , You could check it by running following command

kubectl describe -n kanister repositoryserver.cr.kanister.io/kopia-repo-server-1

9) Create Blueprint

vi mysql-blueprint.yaml

apiVersion: cr.kanister.io/v1alpha1
kind: Blueprint
metadata:
  name: mysql-blueprint
actions:
  backup:
    outputArtifacts:
      mysqlBackup:
        # Capture the kopia snapshot information for subsequent actions
        # The information includes the kopia snapshot ID which is essential for restore and delete to succeed
        # `kopiaOutput` is the name provided to kando using `--output-name` flag
        kopiaSnapshot: "{{ .Phases.dumpToStore.Output.kopiaOutput }}"
    phases:
    - func: KubeTask
      name: dumpToStore
      objects:
        mysqlSecret:
          kind: Secret
          name: '{{ index .Object.metadata.labels "app.kubernetes.io/instance" }}'
          namespace: '{{ .StatefulSet.Namespace }}'
      args:
        image: r4rajat/kanister-tools-custom:v12
        namespace: "{{ .StatefulSet.Namespace }}"
        command:
        - bash
        - -o
        - errexit
        - -o
        - pipefail
        - -c
        - |
          backup_file_path="dump.sql"
          root_password="{{ index .Phases.dumpToStore.Secrets.mysqlSecret.Data "mysql-root-password" | toString }}"
          dump_cmd="mysqldump --column-statistics=0 -u root --password=${root_password} -h {{ index .Object.metadata.labels "app.kubernetes.io/instance" }} --single-transaction --all-databases"
          ${dump_cmd} | kando location push --repository-server '{{ toJson .RepositoryServer }}' --path "${backup_file_path}" --output-name "kopiaOutput" -

  restore:
    inputArtifactNames:
    # The kopia snapshot info created in backup phase can be used here
    # Use the `--kopia-snapshot` flag in kando to pass in `mysqlBackup.KopiaSnapshot`
    - mysqlBackup
    phases:
    - func: KubeTask
      name: restoreFromStore
      objects:
        mysqlSecret:
          kind: Secret
          name: '{{ index .Object.metadata.labels "app.kubernetes.io/instance" }}'
          namespace: '{{ .StatefulSet.Namespace }}'
      args:
        image: r4rajat/kanister-tools-custom:v12
        namespace: "{{ .StatefulSet.Namespace }}"
        command:
        - bash
        - -o
        - errexit
        - -o
        - pipefail
        - -c
        - |
          backup_file_path="dump.sql"
          kopia_snap='{{ .ArtifactsIn.mysqlBackup.KopiaSnapshot }}'
          root_password="{{ index .Phases.restoreFromStore.Secrets.mysqlSecret.Data "mysql-root-password" | toString }}"
          restore_cmd="mysql -u root --password=${root_password} -h {{ index .Object.metadata.labels "app.kubernetes.io/instance" }}"
          kando location pull --repository-server '{{ toJson .RepositoryServer }}' --path "${backup_file_path}" --kopia-snapshot "${kopia_snap}" - | ${restore_cmd}
  delete:
    inputArtifactNames:
    # The kopia snapshot info created in backup phase can be used here
    # Use the `--kopia-snapshot` flag in kando to pass in `mysqlBackup.KopiaSnapshot`
    - mysqlBackup
    phases:
    - func: KubeTask
      name: deleteFromStore
      args:
        image: r4rajat/kanister-tools-custom:v12
        namespace: "{{ .StatefulSet.Namespace }}"
        command:
        - bash
        - -o
        - errexit
        - -o
        - pipefail
        - -c
        - |
          backup_file_path="dump.sql"
          kopia_snap='{{ .ArtifactsIn.mysqlBackup.KopiaSnapshot }}'
          kando location delete --repository-server '{{ toJson .RepositoryServer }}' --path "${backup_file_path}" --kopia-snapshot "${kopia_snap}"

kubectl create -f mysql-blueprint.yaml -n kanister

10) Build kanctl with latest changes

go build -o kanctl cmd/kanctl/main.go

11) Take Backup of the Application

./kanctl create actionset --action backup --namespace kanister --blueprint mysql-blueprint --statefulset mysql/mysql-release --secrets mysql=mysql/mysql-release --repository-server=kopia-repo-server-1

actionset backup-7m5lh created

Check Status of the actionset

kubectl describe actionsets -n kanister backup-7m5lh

Events:
  Type    Reason           Age   From                 Message
  ----    ------           ----  ----                 -------
  Normal  Started Action   9s    Kanister Controller  Executing action backup
  Normal  Started Phase    9s    Kanister Controller  Executing phase dumpToStore
  Normal  Ended Phase      3s    Kanister Controller  Completed phase dumpToStore
  Normal  Update Complete  3s    Kanister Controller  Updated ActionSet 'backup-7m5lh' Status->complete

12) Run Restore Action

./kanctl --namespace kanister create actionset --action restore --from  "backup-7m5lh"

Check status

kubectl describe actionsets -n kanister restore-backup-7m5lh-jeytv

Events:
  Type    Reason           Age   From                 Message
  ----    ------           ----  ----                 -------
  Normal  Started Action   43s   Kanister Controller  Executing action restore
  Normal  Started Phase    11s   Kanister Controller  Executing phase restoreFromStore
  Normal  Ended Phase      5s    Kanister Controller  Completed phase restoreFromStore
  Normal  Update Complete  1s    Kanister Controller  Updated ActionSet 'restore-backup-7m5lh-jeytv' Status->complete

13) Run Delete Action

./kanctl --namespace kanister create actionset --action delete --from  "backup-7m5lh"

Check status

kubectl describe actionsets -n kanister delete-backup-7m5lh-zsqwk

Events:
  Type    Reason           Age   From                 Message
  ----    ------           ----  ----                 -------
  Normal  Started Action   43s   Kanister Controller  Executing action delete
  Normal  Started Phase    11s   Kanister Controller  Executing phase deleteFromStore
  Normal  Ended Phase      5s    Kanister Controller  Completed phase deleteFromStore
  Normal  Update Complete  1s    Kanister Controller  Updated ActionSet 'delete-backup-7m5lh-zsqwk' Status->complete

Verify the delete

Run Restore Action, and check the status, it will throw error that snapshot with id xxxxxxxxxxxx not found