Kopia repository server controller deployment (#1871)

* Expose Azure authenticators + fix typo (#1629) * Expose Azure authenticators * Fix typo * Add support for default msi id by allowing for Azure client ID to exist but be empty * Clean up based on PR suggestions Co-authored-by: Le Tran <le.tran@kasten.io> * Run the triage workflow on primary repo only (#1642) Signed-off-by: Ivan Sim <ivan.sim@kasten.io> * Bump github.com/vmware/govmomi from 0.28.0 to 0.29.0 (#1599) Bumps [github.com/vmware/govmomi](https://github.com/vmware/govmomi) from 0.28.0 to 0.29.0. - [Release notes](https://github.com/vmware/govmomi/releases) - [Changelog](https://github.com/vmware/govmomi/blob/master/CHANGELOG.md) - [Commits](vmware/govmomi@v0.28.0...v0.29.0) --- updated-dependencies: - dependency-name: github.com/vmware/govmomi dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Update the command paths in pkg/kopia/repositoryserver/README.md * Update the README.md * Update the command paths in pkg/kopia/repositoryserver/README.md * Remove go.mod and go.sum files * Remove pkg/kopia/repositoryserver/bin * Removed RepositoryServer CRD manifest and it's client/apis/controller packages from pkg/kopia/repositoryserver * Added RepositoryServer CRD manifest, apis code and client code to pkg/customresource, pkg/apis and pkg/client respectively * Added the symlink to pkg/customresource/repositoryserver.yaml in helm/kanister-operator/crds * Revised the CRD fields * Remove RepositoryServerResource from CreateCustomResources list since it's used by Kanister controller * Add a tag for repositoryserver.yaml from customresource files in pkg/customresource/embed.go * Add new webinar link to README (#1643) Signed-off-by: Ivan Sim <ivan.sim@kasten.io> Signed-off-by: Ivan Sim <ivan.sim@kasten.io> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Remove noStatus local tag from RepositoryServer in types.go to generate UpdateStatus code for it * Add Status subresource to RepositoryServer CRD * Add the 'Ref' suffix to SecretReference CRD fields, update NetworkPolicy labelselector field and replace some omitempty CRD fields to required * Remove kubebuilder specific annotations from CRD YAML * Add Handler and few utility functions to the RepositoryServer controller * Update CRD type and deepcopy function to fix the pod and namespace LabelSelector error * Update controller to use podSelector and namespaceSelector from CR to define ingress traffic for repo server pod * Added Kubebuilder skeleton code * Revert go.sum * Fix vet and fmt target in Kubebuilder.mk * Merge Kubebuilder.mk and Makefile * Remove TODO in Makefile to update the 'test' target name * Rename Kubebuilder build, test, run, deploy and undeploy targets in Makefile * Add isReady field to the CR * Add extra line at the EOF in the YAML * Remove the clientset, listers and informers for RepositoryServer CR * Update RepositoryServer CRD with 'isReady' status field * Regenerate deepcopy methods for the CRs using 'make generate' target * Correct the 'isReady' field in CRD * Remove the previous controller using typed client * Move sample RepositoryServer CR YAML to a different file * Add the new controller implemented using kubebuilder framework * Update the kanister tools image in pod options * Populate pod object using podOptions in a function separate from CreatePod * Update the controller with owned resources and reconciliation for update CR event * Fix the call to GetPodObjectFromPodOptions function in controller handler * Add 'ServerPending' state of RepositoryServer CR in types.go * Set the 'IsReady' field in RepositoryServer CR * Change 'isReady' field to 'Progress' in RepositoryServer CR status * Update the change from 'IsReady' field to 'Progress' field in controller implementation * Add actual status update call to the CR in cluster after updating the CR /status 'Progress' field * Rename podOverrideSpecForCACertificate utility method * Fix infinite error loopback on Update Status * Set the credentials data from secret in the pod as env vars * remove network policy lifecyccle from repository controller * remove network policy lifecycle from the controller * udpate mysql-sidecar dockerfile * address comments * remove prometheus config * repository server start,repository create, add users * remove network policy configuration from CRD * move the server access username to CR and get the hostname and password from secret * move the server access username to CR and get the hostname and password from secret * add server access username changes to repository server CR * Added the manifests for CRD and example CR along with a basic CR controller to log outputs * Run the triage workflow on primary repo only (#1642) Signed-off-by: Ivan Sim <ivan.sim@kasten.io> * Bump github.com/vmware/govmomi from 0.28.0 to 0.29.0 (#1599) Bumps [github.com/vmware/govmomi](https://github.com/vmware/govmomi) from 0.28.0 to 0.29.0. - [Release notes](https://github.com/vmware/govmomi/releases) - [Changelog](https://github.com/vmware/govmomi/blob/master/CHANGELOG.md) - [Commits](vmware/govmomi@v0.28.0...v0.29.0) --- updated-dependencies: - dependency-name: github.com/vmware/govmomi dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Update the command paths in pkg/kopia/repositoryserver/README.md * Update the README.md * Update the command paths in pkg/kopia/repositoryserver/README.md * Remove go.mod and go.sum files * Remove pkg/kopia/repositoryserver/bin * Removed RepositoryServer CRD manifest and it's client/apis/controller packages from pkg/kopia/repositoryserver * Added RepositoryServer CRD manifest, apis code and client code to pkg/customresource, pkg/apis and pkg/client respectively * Added the symlink to pkg/customresource/repositoryserver.yaml in helm/kanister-operator/crds * Revised the CRD fields * Remove RepositoryServerResource from CreateCustomResources list since it's used by Kanister controller * Add a tag for repositoryserver.yaml from customresource files in pkg/customresource/embed.go * Remove noStatus local tag from RepositoryServer in types.go to generate UpdateStatus code for it * Add Status subresource to RepositoryServer CRD * Add the 'Ref' suffix to SecretReference CRD fields, update NetworkPolicy labelselector field and replace some omitempty CRD fields to required * Remove kubebuilder specific annotations from CRD YAML * Update CRD type and deepcopy function to fix the pod and namespace LabelSelector error * Added Kubebuilder skeleton code * Revert go.sum * Fix vet and fmt target in Kubebuilder.mk * Merge Kubebuilder.mk and Makefile * Remove TODO in Makefile to update the 'test' target name * Rename Kubebuilder build, test, run, deploy and undeploy targets in Makefile * Add isReady field to the CR * Add extra line at the EOF in the YAML * Remove the clientset, listers and informers for RepositoryServer CR * Update RepositoryServer CRD with 'isReady' status field * Regenerate deepcopy methods for the CRs using 'make generate' target * Correct the 'isReady' field in CRD * Move sample RepositoryServer CR YAML to a different file * Populate pod object using podOptions in a function separate from CreatePod * Add 'ServerPending' state of RepositoryServer CR in types.go * Change 'isReady' field to 'Progress' in RepositoryServer CR status * remove network policy lifecycle from the controller * address comments * remove prometheus config * remove network policy configuration from CRD * add server access username changes to repository server CR * move the server access username to CR and get the hostname and password from secret * add server access username changes to repository server CR * add repo server controller deployment * go mod tidy * add repository server container in the kanister deployment * remove unwanted kustomization files * sync the GetPodObjectFromPodOptions to master branch * sync the GetPodObjectFromPodOptions to master branch * remove unncessary kustomization.yaml * rebase with crd PR * fix lint issues * remove kustomization files * move context from struct to function variables * move context from struct to function variables * make server start timeout configurable * make repository server start timeout configurable * make server start timeout configurable * go mod tidy * modify copyright information * adding client set for repository server CR * address review comment * address review comments * address review comments * address review comments * move the Makefiles to repository server deployment PR #1871 * add makefile for repository server controller * move the Makefiles to repository server deployment PR #1871 * go mod tidy * address review comments * address review comments * change coyright information * add event filter to skip status update events * make log level configurable, move pod security context to pod options * add event recorder * address review comments * remove kubernetes authors from repositoryserver_types.go * test * add comments to the repository server types * add comments to the repository server types * add comments to the repository server types * address review comments related to inconsistent error/log messages * log messages at debug level * address review comments * go mod tidy * go mod tidy * add log level in values * testing * retrying after sometime * hide repository server controller beyond a flag * remove unnecessary commits * add default enabled flag in values.yaml * add default values in values.yaml * remove private docker registry references * move generate crds to parent makefile * update controller tools version --------- Signed-off-by: Ivan Sim <ivan.sim@kasten.io> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Le Tran <le.u.tran92@gmail.com> Co-authored-by: Le Tran <le.tran@kasten.io> Co-authored-by: Ivan Sim <1330522+ihcsim@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: shlokchaudhari9 <shlokchaudhari9@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
kanisterio · May 3, 2023 · 07038f0 · 07038f0
1 parent 28ccf53
commit 07038f0
Show file tree

Hide file tree

Showing 8 changed files with 154 additions and 0 deletions.
diff --git a/.goreleaser.yml b/.goreleaser.yml
@@ -37,12 +37,24 @@ builds:
   env: *env
   goos: *goos
   goarch: *goarch
+- id: repo-server-controller
+  binary: repo-server-controller
+  main: cmd/reposervercontroller/main.go
+  ldflags: *ldflags
+  env: *env
+  goos: *goos
+  goarch: *goarch
 dockers:
 - ids:
   - controller
   image_templates:
   - 'ghcr.io/kanisterio/controller:{{ .Tag }}'
   dockerfile: 'docker/controller/Dockerfile'
+- ids:
+  - repo-server-controller
+  image_templates:
+  - 'ghcr.io/kanisterio/repo-server-controller:{{ .Tag }}'
+  dockerfile: 'docker/repo-server-controller/Dockerfile'
 - ids:
   - kando
   image_templates:

diff --git a/Makefile b/Makefile
@@ -14,6 +14,10 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
+
+# include repository server's makefile
+include Makefile.kubebuilder
+
 # The binary to build (just the basename).
 BIN := controller
 
@@ -69,6 +73,10 @@ GITHUB_TOKEN ?= ""
 
 GOBORING ?= ""
 
+## Tool Versions
+
+CONTROLLER_TOOLS_VERSION ?= "v0.12.0"
+
 # If you want to build all binaries, see the 'all-build' rule.
 # If you want to build all containers, see the 'all-container' rule.
 # If you want to build AND push all containers, see the 'all-push' rule.
@@ -314,3 +322,14 @@ check:
 
 gomod:
 	@$(MAKE) run CMD='-c "./build/gomod.sh"'
+
+
+install-crds: ## Install CRDs into the K8s cluster specified in ~/.kube/config.
+	@$(MAKE) run CMD='-c "kubectl apply -f pkg/customresource/"'
+
+uninstall-crds: ## Uninstall CRDs from the K8s cluster specified in ~/.kube/config.
+	@$(MAKE) run CMD='-c "kubectl delete -f pkg/customresource/"'
+
+manifests: ## Generates CustomResourceDefinition objects.
+	@$(MAKE) run CMD='-c "./build/generate_crds.sh ${CONTROLLER_TOOLS_VERSION}"'
+
diff --git a/Makefile.kubebuilder b/Makefile.kubebuilder
@@ -0,0 +1,47 @@
+# Copyright 2023 The Kanister Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Image URL to use all building/pushing image targets
+
+# The binary to build (just the basename).
+REPOSERVER_BIN := repo-server-controller
+
+REPOSITORY_SERVER_IMAGE_NAME := repo-server-controller
+REPOSERVER_IMAGE ?= $(REGISTRY)/$(REPOSITORY_SERVER_IMAGE_NAME)
+
+##@ General
+
+# The repo-server-help target prints out all targets with their descriptions organized
+# beneath their categories. The categories are represented by '##@' and the
+# target descriptions by '##'. The awk commands is responsible for reading the
+# entire set of makefiles included in this invocation, looking for lines of the
+# file as xyz: ## something, and then pretty-format the target and help. Then,
+# if there's a line with ##@ something, that gets pretty-printed as a category.
+# More info on the usage of ANSI control characters for terminal formatting:
+# https://en.wikipedia.org/wiki/ANSI_escape_code#SGR_parameters
+# More info on the awk command:
+# http://linuxcommand.org/lc3_adv_awk.php
+
+.PHONY: repo-server-help
+repo-server-help: ## Display the help regarding building and deloying kopia repository server.
+	@awk 'BEGIN {FS = ":.*##"; printf "\nUsage:\n  make \033[36m<target>\033[0m\n"} /^[a-zA-Z_0-9-]+:.*?##/ { printf "  \033[36m%-15s\033[0m %s\n", $$1, $$2 } /^##@/ { printf "\n\033[1m%s\033[0m\n", substr($$0, 5) } ' $(MAKEFILE_LIST)
+
+##@ Build
+
+.PHONY: repo-server-build
+repo-server-build:  ## Build repository server binary.
+	@$(MAKE) run CMD='-c " \
+	goreleaser build --id $(REPOSERVER_BIN) --rm-dist --debug --snapshot \
+	&& cp dist/$(REPOSERVER_BIN)_linux_$(ARCH)/$(REPOSERVER_BIN) bin/$(ARCH)/$(REPOSERVER_BIN) \
+	"'
diff --git a/build/generate_crds.sh b/build/generate_crds.sh
@@ -0,0 +1,31 @@
+#!/bin/bash
+
+# Copyright 2023 The Kanister Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+set -o errexit
+set -o nounset
+set -o xtrace
+
+export GO111MODULE=on
+
+## Location to install dependencies to
+LOCALBIN=$(pwd)/bin
+## Tool Binaries
+CONTROLLER_GEN=${LOCALBIN}/controller-gen
+## Tool Versions
+CONTROLLER_TOOLS_VERSION=${1}
+
+test -s ${CONTROLLER_GEN} || GOBIN=${LOCALBIN} go install sigs.k8s.io/controller-tools/cmd/controller-gen@$(CONTROLLER_TOOLS_VERSION)
+${CONTROLLER_GEN} crd webhook paths="github.com/kanisterio/kanister/pkg/apis/cr/v1alpha1" output:crd:artifacts:config=pkg/customresource
diff --git a/docker/repo-server-controller/Dockerfile b/docker/repo-server-controller/Dockerfile
@@ -0,0 +1,8 @@
+FROM registry.access.redhat.com/ubi9/ubi-minimal:9.1.0-1760
+
+LABEL maintainer="Tom Manville<tom@kasten.io>"
+
+RUN microdnf -y update openssl-libs
+
+ADD repo-server-controller /repo-server-controller
+ENTRYPOINT ["/repo-server-controller"]
diff --git a/helm/kanister-operator/templates/_helpers.tpl b/helm/kanister-operator/templates/_helpers.tpl
@@ -24,6 +24,16 @@ If release name contains chart name it will be used as a full name.
 {{- end -}}
 {{- end -}}
 
+{{/*Create a default repository server controller name
+*/}}
+{{- define "repository-server-controller.name" -}}
+{{- if .Values.repositoryServerController.container.name -}}
+{{- .Values.repositoryServerController.container.name | trunc 63 | trimSuffix "-" -}}
+{{- else -}}
+{{- default "repository-server-controller"}}
+{{- end -}}
+{{- end -}}
+
 {{/*
 Create chart name and version as used by the chart label.
 */}}

diff --git a/helm/kanister-operator/templates/deployment.yaml b/helm/kanister-operator/templates/deployment.yaml
@@ -43,3 +43,17 @@ spec:
         resources:
 {{ toYaml .Values.resources | indent 12 }}
 {{- end }}
+{{- if .Values.repositoryServerController.enabled }}
+      - name: {{ template "repository-server-controller.name" . }}
+        image: {{ .Values.repositoryServerControllerImage.registry }}/{{ .Values.repositoryServerControllerImage.name }}:{{ .Values.repositoryServerControllerImage.tag }}
+        imagePullPolicy: {{ .Values.repositoryServerControllerImage.pullPolicy }}
+        env:
+        - name: KOPIA_SERVER_START_TIMEOUT
+          value: {{ .Values.repositoryServerController.serverStartTimeout | quote }}
+        - name: LOG_LEVEL
+          value: {{ .Values.repositoryServerController.logLevel }}
+{{- if .Values.resources }}
+        resources:
+{{ toYaml .Values.resources | indent 12 }}
+{{- end }}
+{{- end }}
diff --git a/helm/kanister-operator/values.yaml b/helm/kanister-operator/values.yaml
@@ -5,6 +5,11 @@ image:
   repository: ghcr.io/kanisterio/controller
   tag: 0.91.0
   pullPolicy: IfNotPresent
+repositoryServerControllerImage:
+  registry: ghcr.io/kanisterio
+  name: repo-server-controller
+  tag: 0.91.0
+  pullPolicy: IfNotPresent
 rbac:
   create: true
 serviceAccount:
@@ -28,6 +33,14 @@ bpValidatingWebhook:
     mode: auto # If set to `custom` then secretName and caBundle should be provided
     secretName: '' # An already created Secret in kanister controller namespace having tls cert details
     caBundle: '' # A valid, CA bundle which is a PEM-encoded CA bundle for validating the webhook's server certificate
+repositoryServerController:
+  enabled: false
+  # startTimeout is used to specify the time in seconds to wait for starting the kopia repository server
+  # Default timeout is 600 seconds
+  serverStartTimeout: 600
+  logLevel: 'info'
+  container:
+    name: 'repository-server-controller'
 resources:
 # We usually recommend not to specify default resources and to leave this as a conscious
 # choice for the user. This also increases chances charts run on environments with little