Skip to content

Commit

Permalink
Make repository cache settings configurable using repository server CR (
Browse files Browse the repository at this point in the history 
#1918)

* Remove RepositoryServerResource from CreateCustomResources list since it's used by Kanister controller

* Add a tag for repositoryserver.yaml from customresource files in pkg/customresource/embed.go

* Add new webinar link to README (#1643)

Signed-off-by: Ivan Sim <ivan.sim@kasten.io>

Signed-off-by: Ivan Sim <ivan.sim@kasten.io>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Remove noStatus local tag from RepositoryServer in types.go to generate UpdateStatus code for it

* Add Status subresource to RepositoryServer CRD

* Add the 'Ref' suffix to SecretReference CRD fields, update NetworkPolicy labelselector field and replace some omitempty CRD fields to required

* Remove kubebuilder specific annotations from CRD YAML

* Add Handler and few utility functions to the RepositoryServer controller

* Update CRD type and deepcopy function to fix the pod and namespace LabelSelector error

* Update controller to use podSelector and namespaceSelector from CR to define ingress traffic for repo server pod

* Added Kubebuilder skeleton code

* Revert go.sum

* Fix vet and fmt target in Kubebuilder.mk

* Merge Kubebuilder.mk and Makefile

* Remove TODO in Makefile to update the 'test' target name

* Rename Kubebuilder build, test, run, deploy and undeploy targets in Makefile

* Add isReady field to the CR

* Add extra line at the EOF in the YAML

* Remove the clientset, listers and informers for RepositoryServer CR

* Update RepositoryServer CRD with 'isReady' status field

* Regenerate deepcopy methods for the CRs using 'make generate' target

* Correct the 'isReady' field in CRD

* Remove the previous controller using typed client

* Move sample RepositoryServer CR YAML to a different file

* Add the new controller implemented using kubebuilder framework

* Update the kanister tools image in pod options

* Populate pod object using podOptions in a function separate from CreatePod

* Update the controller with owned resources and reconciliation for update CR event

* Fix the call to GetPodObjectFromPodOptions function in controller handler

* Add 'ServerPending' state of RepositoryServer CR in types.go

* Set the 'IsReady' field in RepositoryServer CR

* Change 'isReady' field to 'Progress' in RepositoryServer CR status

* Update the change from 'IsReady' field to 'Progress' field in controller implementation

* Add actual status update call to the CR in cluster after updating the CR /status 'Progress' field

* Rename podOverrideSpecForCACertificate utility method

* Fix infinite error loopback on Update Status

* Set the credentials data from secret in the pod as env vars

* remove network policy lifecyccle from repository controller

* remove network policy lifecycle from the controller

* udpate mysql-sidecar dockerfile

* address comments

* remove prometheus config

* repository server start,repository create, add users

* remove network policy configuration from CRD

* move the server access username to CR and get the hostname and password from secret

* move the server access username to CR and get the hostname and password from secret

* add server access username changes to repository server CR

* Added the manifests for CRD and example CR along with a basic CR controller to log outputs

* Run the triage workflow on primary repo only (#1642)

Signed-off-by: Ivan Sim <ivan.sim@kasten.io>

* Bump github.com/vmware/govmomi from 0.28.0 to 0.29.0 (#1599)

Bumps [github.com/vmware/govmomi](https://github.com/vmware/govmomi) from 0.28.0 to 0.29.0.
- [Release notes](https://github.com/vmware/govmomi/releases)
- [Changelog](https://github.com/vmware/govmomi/blob/master/CHANGELOG.md)
- [Commits](vmware/govmomi@v0.28.0...v0.29.0)

---
updated-dependencies:
- dependency-name: github.com/vmware/govmomi
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Update the command paths in pkg/kopia/repositoryserver/README.md

* Update the README.md

* Update the command paths in pkg/kopia/repositoryserver/README.md

* Remove go.mod and go.sum files

* Remove pkg/kopia/repositoryserver/bin

* Removed RepositoryServer CRD manifest and it's client/apis/controller packages from pkg/kopia/repositoryserver

* Added RepositoryServer CRD manifest, apis code and client code to pkg/customresource, pkg/apis and pkg/client respectively

* Added the symlink to pkg/customresource/repositoryserver.yaml in helm/kanister-operator/crds

* Revised the CRD fields

* Remove RepositoryServerResource from CreateCustomResources list since it's used by Kanister controller

* Add a tag for repositoryserver.yaml from customresource files in pkg/customresource/embed.go

* Remove noStatus local tag from RepositoryServer in types.go to generate UpdateStatus code for it

* Add Status subresource to RepositoryServer CRD

* Add the 'Ref' suffix to SecretReference CRD fields, update NetworkPolicy labelselector field and replace some omitempty CRD fields to required

* Remove kubebuilder specific annotations from CRD YAML

* Update CRD type and deepcopy function to fix the pod and namespace LabelSelector error

* Added Kubebuilder skeleton code

* Revert go.sum

* Fix vet and fmt target in Kubebuilder.mk

* Merge Kubebuilder.mk and Makefile

* Remove TODO in Makefile to update the 'test' target name

* Rename Kubebuilder build, test, run, deploy and undeploy targets in Makefile

* Add isReady field to the CR

* Add extra line at the EOF in the YAML

* Remove the clientset, listers and informers for RepositoryServer CR

* Update RepositoryServer CRD with 'isReady' status field

* Regenerate deepcopy methods for the CRs using 'make generate' target

* Correct the 'isReady' field in CRD

* Move sample RepositoryServer CR YAML to a different file

* Populate pod object using podOptions in a function separate from CreatePod

* Add 'ServerPending' state of RepositoryServer CR in types.go

* Change 'isReady' field to 'Progress' in RepositoryServer CR status

* remove network policy lifecycle from the controller

* address comments

* remove prometheus config

* remove network policy configuration from CRD

* add server access username changes to repository server CR

* move the server access username to CR and get the hostname and password from secret

* add server access username changes to repository server CR

* add repo server controller deployment

* go mod tidy

* add repository server container in the kanister deployment

* remove unwanted kustomization files

* sync the GetPodObjectFromPodOptions to master branch

* sync the GetPodObjectFromPodOptions to master branch

* remove unncessary kustomization.yaml

* rebase with crd PR

* fix lint issues

* remove kustomization files

* move context from struct to function variables

* move context from struct to function variables

* make server start timeout configurable

* make repository server start timeout configurable

* make server start timeout configurable

* go mod tidy

* modify copyright information

* adding client set for repository server CR

* address review comment

* address review comments

* address review comments

* address review comments

* move the Makefiles to repository server deployment PR #1871

* add makefile for repository server controller

* move the Makefiles to repository server deployment PR #1871

* go mod tidy

* address review comments

* address review comments

* change coyright information

* add event filter to skip status update events

* make log level configurable, move pod security context to pod options

* add event recorder

* make repository cache settings configurable using repository server CR

* address review comments

* remove kubernetes authors from repositoryserver_types.go

* test

* add comments to the repository server types

* add comments to the repository server types

* add comments to the repository server types

* address review comments related to inconsistent error/log messages

* log messages at debug level

* address review comments

* address review comments

* go mod tidy

* address review comment

* go mod tidy

* add log level in values

* testing

* retrying after sometime

* hide repository server controller beyond a flag

* remove unnecessary commits

* add default enabled flag in values.yaml

* fix typo

* resolve conflicts

* address review comments

---------

Signed-off-by: Ivan Sim <ivan.sim@kasten.io>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: shlokchaudhari9 <shlokchaudhari9@gmail.com>
Co-authored-by: Ivan Sim <1330522+ihcsim@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Pavan Navarathna <6504783+pavannd1@users.noreply.github.com>
  • Loading branch information
@kale-amruta @shlokc9
@ihcsim @mergify @dependabot @pavannd1
6 people committed May 25, 2023
1 parent 0552645 commit f88b9fa
Show file tree
Hide file tree
Showing 3 changed files with 43 additions and 1 deletion.
8 changes: 8 additions & 0 deletions pkg/apis/cr/v1alpha1/repositoryserver_types.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,14 @@ type Repository struct {
Hostname string `json:"hostname"`
// PasswordSecretRef has the password required to connect to kopia repository
PasswordSecretRef corev1.SecretReference `json:"passwordSecretRef"`
CacheSizeSettings CacheSizeSettings `json:"cacheSizeSettings,omitempty"`
}

// CacheSettings are the metadata/content cache size details
// that can be used while establishing connection to the kopia repository
type CacheSizeSettings struct {
Metadata string `json:"metadata"`
Content string `json:"content"`
}

// Server details required for starting the repository proxy server and initializing the repository client users
Expand Down
24 changes: 23 additions & 1 deletion pkg/controllers/repositoryserver/repository.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@
package repositoryserver

import (
"strconv"

"github.com/kanisterio/kanister/pkg/kopia/command"
"github.com/kanisterio/kanister/pkg/kopia/repository"
)
Expand All @@ -27,7 +29,10 @@ const (
)

func (h *RepoServerHandler) connectToKopiaRepository() error {
contentCacheMB, metadataCacheMB := command.GetGeneralCacheSizeSettings()
contentCacheMB, metadataCacheMB, err := h.getRepositoryCacheSettings()
if err != nil {
return err
}
args := command.RepositoryCommandArgs{
CommandArgs: &command.CommandArgs{
RepoPassword: string(h.RepositoryServerSecrets.repositoryPassword.Data[repoPasswordKey]),
Expand All @@ -52,3 +57,20 @@ func (h *RepoServerHandler) connectToKopiaRepository() error {
args,
)
}

func (h *RepoServerHandler) getRepositoryCacheSettings() (contentCacheMB, metadataCacheMB int, err error) {
contentCacheMB, metadataCacheMB = command.GetGeneralCacheSizeSettings()
if h.RepositoryServer.Spec.Repository.CacheSizeSettings.Content != "" {
contentCacheMB, err = strconv.Atoi(h.RepositoryServer.Spec.Repository.CacheSizeSettings.Content)
if err != nil {
return
}
}
if h.RepositoryServer.Spec.Repository.CacheSizeSettings.Metadata != "" {
metadataCacheMB, err = strconv.Atoi(h.RepositoryServer.Spec.Repository.CacheSizeSettings.Metadata)
if err != nil {
return
}
}
return
}
12 changes: 12 additions & 0 deletions pkg/customresource/repositoryserver.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,18 @@ spec:
description: Repository has the details required by the repository
server to connect to kopia repository
properties:
cacheSizeSettings:
description: CacheSettings are the metadata/content cache size
details that can be used for establishing connection to the kopia repository
properties:
content:
type: string
metadata:
type: string
required:
- content
- metadata
type: object
hostname:
description: If specified, these values will be used by the controller
to override default hostname when connecting to the repository
Expand Down

0 comments on commit f88b9fa

Please sign in to comment.