Helm Chart for HNC installation #354

mochizuki875 · 2024-01-23T08:17:23Z

What type of PR is this?:

/kind feature

What this PR does / why we need it:

Provide HNC installation method via Helm.

Previously, it was disccussed in #46 and #62.
However, the problem that how to keep the chart up-to-date with changes in the config directory has not been resolved.
In this PR, I've added a mechanism to auto generate chart from manifest file generated from config directory using kustomize.

How to use:

Clone this repository.

$ git clone https://github.com/mochizuki875/hierarchical-namespaces.git -b feature_hnc_helm

Set environment variables.
They will be actually set in release process.

$ export HNC_REGISTRY=gcr.io/k8s-staging-multitenancy
$ export HNC_IMG_NAME=hnc-manager
$ export HNC_IMG_TAG=<HNC Version, eg v1.1.0>

Generate chart.

$ make charts

Install chart.
You can do any setting in charts/hnc/values.yaml before installation.
eg: setting hncExcludeNamespaces, enable HRQ, etc.

$ cd charts/hnc
$ helm install hnc ./.  --create-namespace -n hnc-system

Tested:

E2E tests has passed.

Special notes for your reviewer:

Packaging and publishing to chart repository are not included.
In many cases, charts are placed in gh-pages branch and published as Helm repository using GitHub Pages.

eg:
https://github.com/prometheus-community/helm-charts/tree/gh-pages

Onece gh-pages branch is created add published as GitHub Pages, I think we can include these process in cloudbuild.yaml.

mochizuki875 · 2024-01-23T08:18:01Z

/cc @adrianludwin

charts/hnc/templates/hnc-controller-manager-ha.yaml

rjbez17

So sorry, I started a review and got side tracked. Here are a few inital thoughts, I'll find some time soon to give it a closer look. I'm traveling the next few weeks so apologize in advanced.

rjbez17 · 2024-01-29T17:55:11Z

hack/helm_patches/update-helm.sh

+#  limitations under the License.
+
+CURDIR=$(pwd)
+YQ=${CURDIR}/bin/yq


Copy Paste from makefile: I wouldn't expect yq to live in the CURDIR. Maybe add some logic like this instead?

@rjbez17
I may not have understood your comment sufficiently...
update-helm.sh wil be called from Makefile via make charts and yq will be installed in ${CURDIR}/bin before execution update-helm.sh.
Therefore, isn't it expected that yq exists in ${CURDIR}/bin?

hierarchical-namespaces/Makefile

Lines 501 to 505 in b2ccf30

.PHONY: yq

yq: ## Download yq locally if necessary.

@GOBIN=${CURDIR}/bin GO111MODULE=on go install github.com/mikefarah/yq/v4@v4.34.1

charts: yq manifests

rjbez17 · 2024-01-29T17:58:53Z

hack/helm_patches/update-helm.sh

+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+CURDIR=$(pwd)


This is a much safer way to achieve this (ROOTDIR or REPODIR is more descriptive I think):

ROOTDIR="$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )/../"

rjbez17 · 2024-01-29T18:10:31Z

Makefile

+
+charts: yq manifests
+    # Generate CRDs template from manifests/crds.yaml
+	@rm -rf charts/hnc/crds


If you change this to @rm -rf charts/hnc/crds/* you can avoid the mkdir command following it

rjbez17 · 2024-01-29T18:11:13Z

Makefile

+    	cat ${MANIFESTDIR}/crds.yaml | ${YQ} -N -s '.metadata.name + ".yaml"'
+
+    # Generate helm templates from manifests 
+	@rm -rf charts/hnc/templates


rjbez17 · 2024-02-14T20:23:58Z

/ok-to-test

mochizuki875 · 2024-02-22T02:41:32Z

Thanks @rjbez17 !
I've addressed all feedback except one.

rjbez17 · 2024-03-08T21:58:17Z

charts/hnc/values.yaml

@@ -0,0 +1,42 @@
+# Caution: Don't change namespaceOverride from hnc-system.


Is there a reason we don't just hardcode these in _helpers.tpl since the aren't supposed to be overriden?

@rjbez17
As you say, I think it good to hardcode them.
So I've fixed that.

rjbez17 · 2024-03-19T15:33:25Z

Sorry for the delay here. Finally found some time to pull the branch and play with the templating. The ONLY thing left that I can find is can you please make:

hrq:
  enabled: false

true by default?

Building the manifests (make manifests) shows we are enabling by default there and this would keep us inline. Thanks again for all the work here, I really appreciate it.

apiVersion: apps/v1
 kind: Deployment
 metadata:
@@ -789,21 +214,20 @@
     spec:
       containers:
       - args:
+        - --excluded-namespace=hnc-system
+        - --excluded-namespace=kube-system
+        - --excluded-namespace=kube-public
+        - --excluded-namespace=kube-node-lease
         - --webhook-server-port=9443
         - --metrics-addr=:8080
         - --max-reconciles=10
         - --apiserver-qps-throttle=50
-        - --excluded-namespace=kube-system
-        - --excluded-namespace=kube-public
-        - --excluded-namespace=hnc-system
-        - --excluded-namespace=kube-node-lease
         - --nopropagation-label=cattle.io/creator=norman
         - --enable-internal-cert-management
         - --cert-restart-on-secret-refresh
-        - --enable-hrq

mochizuki875 · 2024-03-19T15:44:32Z

@rjbez17
I recognize HRQ is beta status and not enabled by default(default.yaml) at v1.1.0.
As beta feature, it is enabled in hrq.yaml.
https://github.com/kubernetes-sigs/hierarchical-namespaces/releases/tag/v1.1.0

Of course, it's ok to set hrq.enabled: true as helm default value, should we do it?

rjbez17 · 2024-03-19T16:40:30Z

@rjbez17 I recognize HRQ is beta status and not enabled by default(default.yaml) at v1.1.0. As beta feature, it is enabled in hrq.yaml. https://github.com/kubernetes-sigs/hierarchical-namespaces/releases/tag/v1.1.0

Of course, it's ok to set hrq.enabled: true as helm default value, should we do it?

Yeah, I was on the fence with it, but it seems that current manifest generation turns it on by default so I was thinking we'd do the same. We can keep at as false for now since it is an easy change for users to override it on.

rjbez17 · 2024-03-19T16:40:38Z

/lgtm
/approve

k8s-ci-robot · 2024-03-19T16:40:47Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mochizuki875, rjbez17

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [rjbez17]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

mochizuki875 · 2024-03-19T16:44:46Z

@rjbez17 Thanks for your reviews and approve!

mjozefcz · 2024-03-20T08:19:09Z

Thanks guys, I'm really looking forward to see this helm chart as artefact hosted on github pages!

k8s-ci-robot added kind/feature Categorizes issue or PR as related to a new feature. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Jan 23, 2024

k8s-ci-robot requested review from rjbez17 and tashimi January 23, 2024 08:17

k8s-ci-robot requested a review from adrianludwin January 23, 2024 08:18

dcernag reviewed Feb 14, 2024

View reviewed changes

charts/hnc/templates/hnc-controller-manager-ha.yaml Outdated Show resolved Hide resolved

rjbez17 reviewed Feb 14, 2024

View reviewed changes

k8s-ci-robot added the ok-to-test Indicates a non-member PR verified by an org member that is safe to test. label Feb 14, 2024

mochizuki875 force-pushed the feature_hnc_helm branch from b2ccf30 to 676bbdb Compare February 22, 2024 02:28

mochizuki875 changed the title ~~Helm Chart for HNC installation~~ [WIP] Helm Chart for HNC installation Feb 22, 2024

k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 22, 2024

mochizuki875 force-pushed the feature_hnc_helm branch 3 times, most recently from 8c1ae69 to cc1ae7e Compare February 22, 2024 02:38

mochizuki875 changed the title ~~[WIP] Helm Chart for HNC installation~~ Helm Chart for HNC installation Feb 22, 2024

k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 22, 2024

rjbez17 reviewed Mar 8, 2024

View reviewed changes

Add Helm chart

324a67a

mochizuki875 force-pushed the feature_hnc_helm branch from cc1ae7e to 324a67a Compare March 12, 2024 02:26

k8s-ci-robot assigned rjbez17 Mar 19, 2024

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Mar 19, 2024

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 19, 2024

k8s-ci-robot merged commit a0351d2 into kubernetes-sigs:master Mar 19, 2024
3 checks passed

mochizuki875 mentioned this pull request Jun 10, 2024

exclude openshift namespaces via regex #382

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Helm Chart for HNC installation #354

Helm Chart for HNC installation #354

mochizuki875 commented Jan 23, 2024 •

edited

Loading

mochizuki875 commented Jan 23, 2024

rjbez17 left a comment

rjbez17 Jan 29, 2024

mochizuki875 Feb 21, 2024

rjbez17 Jan 29, 2024

rjbez17 Jan 29, 2024

rjbez17 Jan 29, 2024

rjbez17 commented Feb 14, 2024

mochizuki875 commented Feb 22, 2024

rjbez17 Mar 8, 2024

mochizuki875 Mar 12, 2024

rjbez17 commented Mar 19, 2024 •

edited

Loading

mochizuki875 commented Mar 19, 2024

rjbez17 commented Mar 19, 2024

rjbez17 commented Mar 19, 2024

k8s-ci-robot commented Mar 19, 2024

mochizuki875 commented Mar 19, 2024

mjozefcz commented Mar 20, 2024

	.PHONY: yq
	yq: ## Download yq locally if necessary.
	@GOBIN=${CURDIR}/bin GO111MODULE=on go install github.com/mikefarah/yq/v4@v4.34.1

	charts: yq manifests

		@@ -0,0 +1,42 @@
		# Caution: Don't change namespaceOverride from hnc-system.

Helm Chart for HNC installation #354

Helm Chart for HNC installation #354

Conversation

mochizuki875 commented Jan 23, 2024 • edited Loading

What type of PR is this?:

What this PR does / why we need it:

How to use:

Tested:

Related:

Special notes for your reviewer:

mochizuki875 commented Jan 23, 2024

rjbez17 left a comment

Choose a reason for hiding this comment

rjbez17 Jan 29, 2024

Choose a reason for hiding this comment

mochizuki875 Feb 21, 2024

Choose a reason for hiding this comment

rjbez17 Jan 29, 2024

Choose a reason for hiding this comment

rjbez17 Jan 29, 2024

Choose a reason for hiding this comment

rjbez17 Jan 29, 2024

Choose a reason for hiding this comment

rjbez17 commented Feb 14, 2024

mochizuki875 commented Feb 22, 2024

rjbez17 Mar 8, 2024

Choose a reason for hiding this comment

mochizuki875 Mar 12, 2024

Choose a reason for hiding this comment

rjbez17 commented Mar 19, 2024 • edited Loading

mochizuki875 commented Mar 19, 2024

rjbez17 commented Mar 19, 2024

rjbez17 commented Mar 19, 2024

k8s-ci-robot commented Mar 19, 2024

mochizuki875 commented Mar 19, 2024

mjozefcz commented Mar 20, 2024

mochizuki875 commented Jan 23, 2024 •

edited

Loading

rjbez17 commented Mar 19, 2024 •

edited

Loading