Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kopia repository server controller deployment #1871

Merged
merged 269 commits into from
May 3, 2023

Conversation

kale-amruta
Copy link
Contributor

@kale-amruta kale-amruta commented Jan 18, 2023

Kopia repository server deployment

Pull request type

Please check the type of change your PR introduces:

  • 🚧 Work in Progress
  • 🌈 Refactoring (no functional changes, no api changes)
  • 🐹 Trivial/Minor
  • 🐛 Bugfix
  • 🌻 Feature
  • 🗺️ Documentation
  • 🤖 Test

Issues

  • fixes #issue-number

Test Plan

  • 💪 Manual
  • ⚡ Unit test
  • 💚 E2E

leuyentran and others added 30 commits September 30, 2022 13:23
* Expose Azure authenticators

* Fix typo

* Add support for default msi id by allowing for Azure client ID to exist but be empty

* Clean up based on PR suggestions

Co-authored-by: Le Tran <le.tran@kasten.io>
Signed-off-by: Ivan Sim <ivan.sim@kasten.io>
Bumps [github.com/vmware/govmomi](https://github.com/vmware/govmomi) from 0.28.0 to 0.29.0.
- [Release notes](https://github.com/vmware/govmomi/releases)
- [Changelog](https://github.com/vmware/govmomi/blob/master/CHANGELOG.md)
- [Commits](vmware/govmomi@v0.28.0...v0.29.0)

---
updated-dependencies:
- dependency-name: github.com/vmware/govmomi
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…/customresource, pkg/apis and pkg/client respectively
Signed-off-by: Ivan Sim <ivan.sim@kasten.io>

Signed-off-by: Ivan Sim <ivan.sim@kasten.io>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
…anister into kopia-repository-server-controller
…icy labelselector field and replace some omitempty CRD fields to required
…anister into kopia-repository-server-controller
…anister into kopia-repository-server-controller
…anister into kopia-repository-server-controller
@kale-amruta kale-amruta force-pushed the kopia-repository-server-controller-deployment branch from faea463 to ca99a22 Compare April 24, 2023 07:34
Copy link
Contributor

@ankitjain235 ankitjain235 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM besides minor comment.

helm/kanister-operator/values.yaml Outdated Show resolved Hide resolved
docker/repo-server-controller/Dockerfile Show resolved Hide resolved
Copy link
Contributor

@PrasadG193 PrasadG193 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Just have minor comment around if we should switch to latest version of controller-tools.

Makefile Outdated Show resolved Hide resolved
@kale-amruta kale-amruta force-pushed the kopia-repository-server-controller-deployment branch from 2067e84 to 8d28549 Compare May 3, 2023 07:55
@mergify mergify bot merged commit 07038f0 into master May 3, 2023
@mergify mergify bot deleted the kopia-repository-server-controller-deployment branch May 3, 2023 08:38
mergify bot added a commit that referenced this pull request May 25, 2023
#1918)

* Remove RepositoryServerResource from CreateCustomResources list since it's used by Kanister controller

* Add a tag for repositoryserver.yaml from customresource files in pkg/customresource/embed.go

* Add new webinar link to README (#1643)

Signed-off-by: Ivan Sim <ivan.sim@kasten.io>

Signed-off-by: Ivan Sim <ivan.sim@kasten.io>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Remove noStatus local tag from RepositoryServer in types.go to generate UpdateStatus code for it

* Add Status subresource to RepositoryServer CRD

* Add the 'Ref' suffix to SecretReference CRD fields, update NetworkPolicy labelselector field and replace some omitempty CRD fields to required

* Remove kubebuilder specific annotations from CRD YAML

* Add Handler and few utility functions to the RepositoryServer controller

* Update CRD type and deepcopy function to fix the pod and namespace LabelSelector error

* Update controller to use podSelector and namespaceSelector from CR to define ingress traffic for repo server pod

* Added Kubebuilder skeleton code

* Revert go.sum

* Fix vet and fmt target in Kubebuilder.mk

* Merge Kubebuilder.mk and Makefile

* Remove TODO in Makefile to update the 'test' target name

* Rename Kubebuilder build, test, run, deploy and undeploy targets in Makefile

* Add isReady field to the CR

* Add extra line at the EOF in the YAML

* Remove the clientset, listers and informers for RepositoryServer CR

* Update RepositoryServer CRD with 'isReady' status field

* Regenerate deepcopy methods for the CRs using 'make generate' target

* Correct the 'isReady' field in CRD

* Remove the previous controller using typed client

* Move sample RepositoryServer CR YAML to a different file

* Add the new controller implemented using kubebuilder framework

* Update the kanister tools image in pod options

* Populate pod object using podOptions in a function separate from CreatePod

* Update the controller with owned resources and reconciliation for update CR event

* Fix the call to GetPodObjectFromPodOptions function in controller handler

* Add 'ServerPending' state of RepositoryServer CR in types.go

* Set the 'IsReady' field in RepositoryServer CR

* Change 'isReady' field to 'Progress' in RepositoryServer CR status

* Update the change from 'IsReady' field to 'Progress' field in controller implementation

* Add actual status update call to the CR in cluster after updating the CR /status 'Progress' field

* Rename podOverrideSpecForCACertificate utility method

* Fix infinite error loopback on Update Status

* Set the credentials data from secret in the pod as env vars

* remove network policy lifecyccle from repository controller

* remove network policy lifecycle from the controller

* udpate mysql-sidecar dockerfile

* address comments

* remove prometheus config

* repository server start,repository create, add users

* remove network policy configuration from CRD

* move the server access username to CR and get the hostname and password from secret

* move the server access username to CR and get the hostname and password from secret

* add server access username changes to repository server CR

* Added the manifests for CRD and example CR along with a basic CR controller to log outputs

* Run the triage workflow on primary repo only (#1642)

Signed-off-by: Ivan Sim <ivan.sim@kasten.io>

* Bump github.com/vmware/govmomi from 0.28.0 to 0.29.0 (#1599)

Bumps [github.com/vmware/govmomi](https://github.com/vmware/govmomi) from 0.28.0 to 0.29.0.
- [Release notes](https://github.com/vmware/govmomi/releases)
- [Changelog](https://github.com/vmware/govmomi/blob/master/CHANGELOG.md)
- [Commits](vmware/govmomi@v0.28.0...v0.29.0)

---
updated-dependencies:
- dependency-name: github.com/vmware/govmomi
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Update the command paths in pkg/kopia/repositoryserver/README.md

* Update the README.md

* Update the command paths in pkg/kopia/repositoryserver/README.md

* Remove go.mod and go.sum files

* Remove pkg/kopia/repositoryserver/bin

* Removed RepositoryServer CRD manifest and it's client/apis/controller packages from pkg/kopia/repositoryserver

* Added RepositoryServer CRD manifest, apis code and client code to pkg/customresource, pkg/apis and pkg/client respectively

* Added the symlink to pkg/customresource/repositoryserver.yaml in helm/kanister-operator/crds

* Revised the CRD fields

* Remove RepositoryServerResource from CreateCustomResources list since it's used by Kanister controller

* Add a tag for repositoryserver.yaml from customresource files in pkg/customresource/embed.go

* Remove noStatus local tag from RepositoryServer in types.go to generate UpdateStatus code for it

* Add Status subresource to RepositoryServer CRD

* Add the 'Ref' suffix to SecretReference CRD fields, update NetworkPolicy labelselector field and replace some omitempty CRD fields to required

* Remove kubebuilder specific annotations from CRD YAML

* Update CRD type and deepcopy function to fix the pod and namespace LabelSelector error

* Added Kubebuilder skeleton code

* Revert go.sum

* Fix vet and fmt target in Kubebuilder.mk

* Merge Kubebuilder.mk and Makefile

* Remove TODO in Makefile to update the 'test' target name

* Rename Kubebuilder build, test, run, deploy and undeploy targets in Makefile

* Add isReady field to the CR

* Add extra line at the EOF in the YAML

* Remove the clientset, listers and informers for RepositoryServer CR

* Update RepositoryServer CRD with 'isReady' status field

* Regenerate deepcopy methods for the CRs using 'make generate' target

* Correct the 'isReady' field in CRD

* Move sample RepositoryServer CR YAML to a different file

* Populate pod object using podOptions in a function separate from CreatePod

* Add 'ServerPending' state of RepositoryServer CR in types.go

* Change 'isReady' field to 'Progress' in RepositoryServer CR status

* remove network policy lifecycle from the controller

* address comments

* remove prometheus config

* remove network policy configuration from CRD

* add server access username changes to repository server CR

* move the server access username to CR and get the hostname and password from secret

* add server access username changes to repository server CR

* add repo server controller deployment

* go mod tidy

* add repository server container in the kanister deployment

* remove unwanted kustomization files

* sync the GetPodObjectFromPodOptions to master branch

* sync the GetPodObjectFromPodOptions to master branch

* remove unncessary kustomization.yaml

* rebase with crd PR

* fix lint issues

* remove kustomization files

* move context from struct to function variables

* move context from struct to function variables

* make server start timeout configurable

* make repository server start timeout configurable

* make server start timeout configurable

* go mod tidy

* modify copyright information

* adding client set for repository server CR

* address review comment

* address review comments

* address review comments

* address review comments

* move the Makefiles to repository server deployment PR #1871

* add makefile for repository server controller

* move the Makefiles to repository server deployment PR #1871

* go mod tidy

* address review comments

* address review comments

* change coyright information

* add event filter to skip status update events

* make log level configurable, move pod security context to pod options

* add event recorder

* make repository cache settings configurable using repository server CR

* address review comments

* remove kubernetes authors from repositoryserver_types.go

* test

* add comments to the repository server types

* add comments to the repository server types

* add comments to the repository server types

* address review comments related to inconsistent error/log messages

* log messages at debug level

* address review comments

* address review comments

* go mod tidy

* address review comment

* go mod tidy

* add log level in values

* testing

* retrying after sometime

* hide repository server controller beyond a flag

* remove unnecessary commits

* add default enabled flag in values.yaml

* fix typo

* resolve conflicts

* address review comments

---------

Signed-off-by: Ivan Sim <ivan.sim@kasten.io>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: shlokchaudhari9 <shlokchaudhari9@gmail.com>
Co-authored-by: Ivan Sim <1330522+ihcsim@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Pavan Navarathna <6504783+pavannd1@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants