From 4e65c821d583898cdba83537cda2fa5dc70e2762 Mon Sep 17 00:00:00 2001 From: Roman Dodin Date: Fri, 29 Jul 2022 11:05:03 +0200 Subject: [PATCH 01/18] updated readme --- README.md | 62 ++++++++++++++++++++++--------------------------------- 1 file changed, 25 insertions(+), 37 deletions(-) diff --git a/README.md b/README.md index 1fd30d3..1b197a0 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,11 @@ +# SR Linux Controller for KNE + This is a k8s controller for running and managing SR Linux nodes launched from [openconfig/kne](https://github.com/openconfig/kne) topology. ## Install + To install the latest version of this controller on a cluster referenced in `~/.kube/config` issue the following command: + ```bash # latest version kubectl apply -k https://github.com/srl-labs/srl-controller/config/default @@ -11,7 +15,8 @@ kubectl apply -k https://github.com/srl-labs/srl-controller/config/default?ref=v ``` The resources of this controller will be scoped under `srlinux-controller` namespace. -``` + +```text ❯ kubectl get all -n srlinux-controller NAME READY STATUS RESTARTS AGE @@ -28,86 +33,69 @@ replicaset.apps/srlinux-controller-controller-manager-c7495dcc7 1 1 ``` ### Installing from a repo + The controller can be installed with make directly from the repo: -``` + +```text make deploy IMG=ghcr.io/srl-labs/srl-controller:0.3.1 ``` -Make sure to check which controller versions are [available](https://github.com/srl-labs/srl-controller/pkgs/container/srl-controller/versions) +Make sure to check which controller versions are [available](https://github.com/srl-labs/srl-controller/pkgs/container/srl-controller/versions). ## Uninstall + To uninstall the controller from the cluster: -``` + +```text kubectl delete -k https://github.com/srl-labs/srl-controller/config/default ``` -## Testing with `kind` -To run this controller in a test cluster deployed with [`kind`](https://kind.sigs.k8s.io/) follow the steps outlined below. +## Testing with `kne` & `kind` -1. Install `kind` -2. Clone and enter into [openconfig/kne](https://github.com/openconfig/kne) repo. -3. Build the kne cli with - `cd kne_cli && go build -o kne && chmod +x ./kne && mv ./kne /usr/local/bin` -4. deploy kind cluster and the necessary CNI with `kne deploy deploy/kne/kind.yaml` where the path to `kind.yaml` is a relative path from the root of the kne repo. +To run this controller in a test cluster deployed with [`kne`](https://github.com/openconfig/kne) and [`kind`](https://kind.sigs.k8s.io/) follow the steps outlined in the [KNE repository](https://github.com/openconfig/kne/tree/main/docs). -This will install the `kind` cluster named `kne` with [`meshnet-cni`](https://github.com/networkop/meshnet-cni) and [`metallb`](https://metallb.universe.tf/). - -A quick test may be performed to verify that SR Linux pods can communicate over the newly deployed cluster: - -```bash -# apply this manifest https://gist.github.com/hellt/43cfade6178be32ea7dfa5cb64715822 -# which has two srlinux pods and two linux pods -kubectl apply -f https://gist.githubusercontent.com/hellt/43cfade6178be32ea7dfa5cb64715822/raw/847a10a57dca996432be7c4a9743c0e0c5b75814/srl.yml - -# once the pods are deployed and running, verify that srlinux pods can reach each other with ssh -# check what IP the srl2 has -kubectl exec -it srl1 -- ip netns exec srbase-mgmt ssh admin@10.244.0.7 -admin@10.244.0.7's password: -Last login: Fri Sep 24 13:58:05 2021 from 10.244.0.6 -Using configuration file(s): ['/etc/opt/srlinux/srlinux.rc'] -Welcome to the srlinux CLI. -Type 'help' (and press ) if you need any help using this. ---{ [FACTORY] running }--[ ]-- -``` - -If all works as expected a [demo topology with three SR Linux nodes](https://github.com/openconfig/kne/blob/main/examples/3node-srl.pb.txt) may be deployed as follows: +Once the kne+kind cluster is created, a [demo topology with two SR Linux nodes](https://github.com/openconfig/kne/blob/db5fe5be01a1b6b65bd79e740e2c819c5aeb50b0/examples/srlinux/2node-srl-with-config.pbtxt) may be deployed as follows: ```bash -kne create ~/kne/examples/3node-srl.pb.txt +kne create examples/srlinux/2node-srl-with-config.pbtxt ``` This will deploy the SR Linux nodes and will create k8s services as per the topology configuration. The services will be exposed via MetalLB and can be queried as: -``` +```text ❯ kubectl -n 3node-srlinux get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service-r1 LoadBalancer 10.96.151.84 172.19.0.50 57400:30006/TCP,443:30004/TCP,22:30005/TCP 6m10s service-r2 LoadBalancer 10.96.34.36 172.19.0.51 443:30010/TCP,22:30011/TCP,57400:30009/TCP 6m9s -service-r3 LoadBalancer 10.96.159.220 172.19.0.52 443:30015/TCP,22:30016/TCP,57400:30014/TCP 6m9s ``` To connect with SSH to the `r1` node, use `ssh admin@172.19.0.50` command. ## Controller operations + The controller is designed to manage the `Srlinux` custom resource defined with [the following CRD](https://doc.crds.dev/github.com/srl-labs/srl-controller). The request to create/delete a resource of kind `Srlinux` is typically coming from `openconfig/kne` topology. ### Creation + When a request to create an `Srlinux` resource named `r1` in namespace `ns` comes in, the controller's reconcile loop does the following: -1. Checks if the pods exist within a namespace `ns` with a name `r1` +1. Checks if the pods exist within a namespace `ns` with a name `r1` 2. If the pod hasn't been found, then the controller first ensures that the necessary config maps exist in namespace `ns` and creates them otherwise. 3. When config maps are sorted out, the controller schedules a pod with the name `r1` and requeue the request 4. In a requeue run, the pod is now found and the controller updates the status of `Srlinux` resource with the image name that was used in the pod spec. ### Deletion + When a deletion happens on `Srlinux` resource, the reconcile loop does nothing. ### API access + This repo contains a clientset for API access to the `Srlinux` custom resource. Check [kne repo](https://github.com/openconfig/kne/blob/fc195a73035bcbf344791979ca3e067be47a249c/topo/node/srl/srl.go#L46) to see how this can be done. ## Building srl-controller container image + To build `srl-controller` container image execute: ```bash @@ -115,4 +103,4 @@ To build `srl-controller` container image execute: make docker-build IMG=ghcr.io/srl-labs/srl-controller:${tag} ``` -Then upload the image to the registry. \ No newline at end of file +Then upload the image to the registry. From 4fc95e54d105cac40e3b8b35dddc6c98f40dba97 Mon Sep 17 00:00:00 2001 From: Roman Dodin Date: Fri, 29 Jul 2022 11:51:46 +0200 Subject: [PATCH 02/18] updated controller version in kustomization --- README.md | 2 +- config/manager/kustomization.yaml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 1b197a0..c412102 100644 --- a/README.md +++ b/README.md @@ -103,4 +103,4 @@ To build `srl-controller` container image execute: make docker-build IMG=ghcr.io/srl-labs/srl-controller:${tag} ``` -Then upload the image to the registry. +Then upload the image to the registry and update the controller version in [manager/kustomization.yaml](config/manager/kustomization.yaml). diff --git a/config/manager/kustomization.yaml b/config/manager/kustomization.yaml index 7b71911..f617449 100644 --- a/config/manager/kustomization.yaml +++ b/config/manager/kustomization.yaml @@ -13,4 +13,4 @@ kind: Kustomization images: - name: controller newName: ghcr.io/srl-labs/srl-controller - newTag: 0.3.1 + newTag: 0.3.4 From 7841a97c596d4bd854f0f4ec8a7fcca3320023dd Mon Sep 17 00:00:00 2001 From: Roman Dodin Date: Fri, 29 Jul 2022 12:01:36 +0200 Subject: [PATCH 03/18] added lint cmds for local use --- .mk/lint.mk | 5 ++++- README.md | 2 +- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/.mk/lint.mk b/.mk/lint.mk index 6a1ad93..c9ff20a 100644 --- a/.mk/lint.mk +++ b/.mk/lint.mk @@ -7,13 +7,16 @@ GODOT_FLAGS := -w . GOLINES_CMD := docker run --rm -it -v $(shell pwd):/work ghcr.io/hellt/golines:0.10.0 golines GOLINES_FLAGS := -w . -GOLANGCI_CMD := docker run -it --rm -v $$(pwd):/app -w /app golangci/golangci-lint:v1.47.2 golangci-lint +GOLANGCI_CMD := docker run -it --rm -v $(shell pwd):/app -w /app golangci/golangci-lint:v1.47.2 golangci-lint GOLANGCI_FLAGS := --config ./.github/workflows/linters/.golangci.yml run -v --fix # when running in a CI env we use locally installed bind ifdef CI GOFUMPT_CMD := gofumpt + GODOT_CMD := godot + GOLINES_CMD := golines + GOLANGCI_CMD := golangci-lint endif diff --git a/README.md b/README.md index c412102..175bec8 100644 --- a/README.md +++ b/README.md @@ -79,7 +79,7 @@ The request to create/delete a resource of kind `Srlinux` is typically coming fr ### Creation -When a request to create an `Srlinux` resource named `r1` in namespace `ns` comes in, the controller's reconcile loop does the following: +When a request to create a `Srlinux` resource named `r1` in namespace `ns` comes in, the controller's reconcile loop does the following: 1. Checks if the pods exist within a namespace `ns` with a name `r1` 2. If the pod hasn't been found, then the controller first ensures that the necessary config maps exist in namespace `ns` and creates them otherwise. From 490a01142ee7f7eeb2a031b774d5dc1dbd36786c Mon Sep 17 00:00:00 2001 From: Roman Dodin Date: Fri, 29 Jul 2022 13:32:04 +0200 Subject: [PATCH 04/18] added 6e/10e variants --- manifests/variants/srl_variants.yml | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/manifests/variants/srl_variants.yml b/manifests/variants/srl_variants.yml index c8feb6f..30a0f51 100644 --- a/manifests/variants/srl_variants.yml +++ b/manifests/variants/srl_variants.yml @@ -99,6 +99,18 @@ data: "card_type": 127 "mda_type": 36 + ixr6e: | + # ixr6e + chassis_configuration: + "chassis_type": 68 + "base_mac": 02:__RANDMAC__:00:00:00 + "cpm_card_type": 184 + + slot_configuration: + 1: + "card_type": 182 + "mda_type": 199 + ixr10: | # ixr10 chassis_configuration: @@ -110,3 +122,15 @@ data: 1: "card_type": 127 "mda_type": 36 + + ixr10e: | + # ixr10e + chassis_configuration: + "chassis_type": 69 + "base_mac": 02:__RANDMAC__:00:00:00 + "cpm_card_type": 184 + + slot_configuration: + 1: + "card_type": 182 + "mda_type": 199 From a1335934a2138d775dfec55a24f2df9cdc86c0e3 Mon Sep 17 00:00:00 2001 From: Roman Dodin Date: Fri, 29 Jul 2022 13:32:17 +0200 Subject: [PATCH 05/18] removed extra shebang --- manifests/variants/kne-entrypoint.yml | 2 -- 1 file changed, 2 deletions(-) diff --git a/manifests/variants/kne-entrypoint.yml b/manifests/variants/kne-entrypoint.yml index 8748ac7..16276b9 100644 --- a/manifests/variants/kne-entrypoint.yml +++ b/manifests/variants/kne-entrypoint.yml @@ -9,7 +9,5 @@ data: #!/bin/bash # this entrypoint ensures that we call topomac script before executing the main entrypoint - #!/bin/bash - sudo bash /tmp/topomac/topomac.sh exec /entrypoint.sh "$@" From f87a4f8c289969f62a9f558354ae537b873c04d9 Mon Sep 17 00:00:00 2001 From: Roman Dodin Date: Fri, 29 Jul 2022 13:32:24 +0200 Subject: [PATCH 06/18] added private to ignore --- .gitignore | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/.gitignore b/.gitignore index a25b34c..50b68e1 100644 --- a/.gitignore +++ b/.gitignore @@ -1,4 +1,3 @@ - # Binaries for programs and plugins *.exe *.exe~ @@ -24,3 +23,5 @@ testbin/* *.swo *~ .vscode + +private From e47678ac592be4b3d08c162213ef81883b80d259 Mon Sep 17 00:00:00 2001 From: Roman Dodin Date: Fri, 29 Jul 2022 14:14:50 +0200 Subject: [PATCH 07/18] carve out volume mounts creation --- controllers/pod.go | 34 +++++++++++++++++-------------- controllers/srlinux_controller.go | 2 +- 2 files changed, 20 insertions(+), 16 deletions(-) diff --git a/controllers/pod.go b/controllers/pod.go index 413a1bc..ec6390e 100644 --- a/controllers/pod.go +++ b/controllers/pod.go @@ -85,21 +85,7 @@ func createContainers(s *typesv1a1.Srlinux) []corev1.Container { Privileged: pointer.Bool(true), RunAsUser: pointer.Int64(0), }, - VolumeMounts: []corev1.VolumeMount{ - { - Name: variantsVolName, - MountPath: variantsVolMntPath, - }, - { - Name: topomacVolName, - MountPath: topomacVolMntPath, - }, - { - Name: entrypointVolName, - MountPath: entrypointVolMntPath, - SubPath: entrypointVolMntSubPath, - }, - }, + VolumeMounts: createVolumeMounts(), }} } @@ -210,3 +196,21 @@ func handleStartupConfig(s *typesv1a1.Srlinux, pod *corev1.Pod, log logr.Logger) ) } } + +func createVolumeMounts() []corev1.VolumeMount { + return []corev1.VolumeMount{ + { + Name: variantsVolName, + MountPath: variantsVolMntPath, + }, + { + Name: topomacVolName, + MountPath: topomacVolMntPath, + }, + { + Name: entrypointVolName, + MountPath: entrypointVolMntPath, + SubPath: entrypointVolMntSubPath, + }, + } +} diff --git a/controllers/srlinux_controller.go b/controllers/srlinux_controller.go index 32d952f..6162174 100644 --- a/controllers/srlinux_controller.go +++ b/controllers/srlinux_controller.go @@ -47,7 +47,7 @@ const ( entrypointVolName = "kne-entrypoint" entrypointVolMntPath = "/kne-entrypoint.sh" - entrypointVolMntSubPath = "kne-entrypoint.sh" + entrypointVolMntSubPath = "kne-entrypoint.sh" // used to enable setting file permissions for a file entrypointCfgMapName = "srlinux-kne-entrypoint" // default path to a startup config file From a205161e2d984a973e8679d2b24aa19595937224 Mon Sep 17 00:00:00 2001 From: Roman Dodin Date: Fri, 29 Jul 2022 22:29:53 +0200 Subject: [PATCH 08/18] remove testpackage linter --- .github/workflows/linters/.golangci.yml | 1 - 1 file changed, 1 deletion(-) diff --git a/.github/workflows/linters/.golangci.yml b/.github/workflows/linters/.golangci.yml index 25836d5..8646bab 100644 --- a/.github/workflows/linters/.golangci.yml +++ b/.github/workflows/linters/.golangci.yml @@ -58,7 +58,6 @@ linters: - goerr113 - nestif - prealloc - - testpackage - wsl linters-settings: lll: From 1d8062156a4fe1f39e86eb231e895b71adebac59 Mon Sep 17 00:00:00 2001 From: Roman Dodin Date: Fri, 29 Jul 2022 22:30:21 +0200 Subject: [PATCH 09/18] added namespace kustomizatioin --- config/namespace/kustomization.yml | 6 ++++++ config/namespace/namespace.yml | 6 ++++++ 2 files changed, 12 insertions(+) create mode 100644 config/namespace/kustomization.yml create mode 100644 config/namespace/namespace.yml diff --git a/config/namespace/kustomization.yml b/config/namespace/kustomization.yml new file mode 100644 index 0000000..6235606 --- /dev/null +++ b/config/namespace/kustomization.yml @@ -0,0 +1,6 @@ +# this namespace resource is created separately to support +# license secrets to be created with a namespace provisioned automatically +# if srl controller is created first, then this namespace will be already created +# as part of the manager kustomization. +resources: + - "namespace.yml" diff --git a/config/namespace/namespace.yml b/config/namespace/namespace.yml new file mode 100644 index 0000000..8b55c3c --- /dev/null +++ b/config/namespace/namespace.yml @@ -0,0 +1,6 @@ +apiVersion: v1 +kind: Namespace +metadata: + labels: + control-plane: controller-manager + name: system From e914880b1e02f299cc46b8ef31c43f90dbea3d24 Mon Sep 17 00:00:00 2001 From: Roman Dodin Date: Fri, 29 Jul 2022 22:41:48 +0200 Subject: [PATCH 10/18] added GetImage tests --- api/types/v1alpha1/srlinux_types.go | 22 ++++++---- api/types/v1alpha1/srlinux_types_test.go | 55 ++++++++++++++++++++++++ go.mod | 3 +- go.sum | 3 ++ 4 files changed, 74 insertions(+), 9 deletions(-) create mode 100644 api/types/v1alpha1/srlinux_types_test.go diff --git a/api/types/v1alpha1/srlinux_types.go b/api/types/v1alpha1/srlinux_types.go index 70e3aa7..d75f79f 100644 --- a/api/types/v1alpha1/srlinux_types.go +++ b/api/types/v1alpha1/srlinux_types.go @@ -23,12 +23,14 @@ import ( // SrlinuxSpec defines the desired state of Srlinux. type SrlinuxSpec struct { // Important: Run "make" to regenerate code after modifying this file - Config *NodeConfig `json:"config,omitempty"` NumInterfaces int `json:"num-interfaces,omitempty"` Constraints map[string]string `json:"constraints,omitempty"` - Model string `json:"model,omitempty"` - Version string `json:"version,omitempty"` + // Model encodes SR Linux variant (ixr-d3, ixr-6e, etc) + Model string `json:"model,omitempty"` + // Version may be set in kne topology as a mean to explicitly provide version information + // in case it is not encoded in the image tag + Version string `json:"version,omitempty"` } // SrlinuxStatus defines the observed state of Srlinux. @@ -70,7 +72,7 @@ func (s *SrlinuxSpec) GetConfig() *NodeConfig { return s.Config } - return nil + return &NodeConfig{} } // GetConstraints gets constraints from srlinux spec, @@ -97,13 +99,17 @@ func (s *SrlinuxSpec) GetModel() string { // if Config.Image is provided it takes precedence over all other option // if not, the Spec.Version is used as a tag for public container image ghcr.io/nokia/srlinux. func (s *SrlinuxSpec) GetImage() string { + img := defaultSRLinuxImageName + if s.GetConfig().Image != "" { - return s.GetConfig().Image + img = s.GetConfig().Image } - if s.Version != "" { - return defaultSRLinuxImageName + ":" + s.Version + // when image is not defined, but version is + // the version is used as a tag for a default image repo + if s.GetConfig().Image == "" && s.Version != "" { + img = img + ":" + s.Version } - return defaultSRLinuxImageName + return img } diff --git a/api/types/v1alpha1/srlinux_types_test.go b/api/types/v1alpha1/srlinux_types_test.go new file mode 100644 index 0000000..afdebca --- /dev/null +++ b/api/types/v1alpha1/srlinux_types_test.go @@ -0,0 +1,55 @@ +package v1alpha1 + +import ( + "testing" + + "github.com/google/go-cmp/cmp" +) + +func TestGetImage(t *testing.T) { + + tests := []struct { + desc string + spec *SrlinuxSpec + want string + }{ + { + desc: "no image, no version, default applies", + spec: &SrlinuxSpec{}, + want: defaultSRLinuxImageName, + }, + { + desc: "image with valid tag", + spec: &SrlinuxSpec{ + Config: &NodeConfig{ + Image: "ghcr.io/nokia/srlinux:22.6.1", + }, + }, + want: "ghcr.io/nokia/srlinux:22.6.1", + }, + { + desc: "image undefined, version present", + spec: &SrlinuxSpec{ + Version: "21.11.1", + }, + want: defaultSRLinuxImageName + ":21.11.1", + }, + } + + for _, tt := range tests { + t.Run(tt.desc, func(t *testing.T) { + img := tt.spec.GetImage() + + if !cmp.Equal(img, tt.want) { + t.Fatalf( + "%s: actual and expected inputs do not match\nactual: %+v\nexpected:%+v", + tt.desc, + img, + tt.want, + ) + } + }, + ) + } + +} diff --git a/go.mod b/go.mod index dc0f234..416cffa 100644 --- a/go.mod +++ b/go.mod @@ -32,7 +32,7 @@ require ( github.com/gogo/protobuf v1.3.2 // indirect github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e // indirect github.com/golang/protobuf v1.5.2 // indirect - github.com/google/go-cmp v0.5.7 // indirect + github.com/google/go-cmp v0.5.8 // indirect github.com/google/gofuzz v1.2.0 // indirect github.com/google/uuid v1.2.0 // indirect github.com/googleapis/gnostic v0.5.5 // indirect @@ -70,6 +70,7 @@ require ( gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 // indirect gopkg.in/yaml.v2 v2.4.0 // indirect gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b // indirect + gotest.tools v2.2.0+incompatible // indirect k8s.io/apiextensions-apiserver v0.21.2 // indirect k8s.io/component-base v0.21.2 // indirect k8s.io/klog/v2 v2.8.0 // indirect diff --git a/go.sum b/go.sum index a04452e..3ab08aa 100644 --- a/go.sum +++ b/go.sum @@ -382,6 +382,8 @@ github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/ github.com/google/go-cmp v0.5.6/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.7 h1:81/ik6ipDQS2aGcBfIN5dHDB36BwrStyeAQquSYCV4o= github.com/google/go-cmp v0.5.7/go.mod h1:n+brtR0CgQNWTVd5ZUFpTBC8YFBDLK/h/bpaJ8/DtOE= +github.com/google/go-cmp v0.5.8 h1:e6P7q2lk1O+qJJb4BtCQXlK8vWEO8V1ZeuEdJNOqZyg= +github.com/google/go-cmp v0.5.8/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= github.com/google/goexpect v0.0.0-20210430020637-ab937bf7fd6f/go.mod h1:n1ej5+FqyEytMt/mugVDZLIiqTMO+vsrgY+kM6ohzN0= github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= github.com/google/gofuzz v1.1.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= @@ -1164,6 +1166,7 @@ gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C gopkg.in/yaml.v3 v3.0.0-20200615113413-eeeca48fe776/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b h1:h8qDotaEPuJATrMmW04NCwg7v22aHH28wwpauUhK9Oo= gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= +gotest.tools v2.2.0+incompatible h1:VsBPFP1AI068pPrMxtb/S8Zkgf9xEmTLJjfM+P5UIEo= gotest.tools v2.2.0+incompatible/go.mod h1:DsYFclhRJ6vuDpmuTbkuFWG+y2sxOXAzmJt81HFBacw= gotest.tools/v3 v3.0.2/go.mod h1:3SzNCllyD9/Y+b5r9JIKQ474KzkZyqLqEfYqMsX94Bk= gotest.tools/v3 v3.0.3/go.mod h1:Z7Lb0S5l+klDB31fvDQX8ss/FlKDxtlFlw3Oa8Ymbl8= From 1bfe4145cd28d06c67940ffc849a6b3b4caa587e Mon Sep 17 00:00:00 2001 From: Roman Dodin Date: Fri, 29 Jul 2022 22:44:40 +0200 Subject: [PATCH 11/18] added crd comments --- api/types/v1alpha1/srlinux_types.go | 1 - config/crd/bases/kne.srlinux.dev_srlinuxes.yaml | 4 ++++ 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/api/types/v1alpha1/srlinux_types.go b/api/types/v1alpha1/srlinux_types.go index d75f79f..549c6e7 100644 --- a/api/types/v1alpha1/srlinux_types.go +++ b/api/types/v1alpha1/srlinux_types.go @@ -22,7 +22,6 @@ import ( // SrlinuxSpec defines the desired state of Srlinux. type SrlinuxSpec struct { - // Important: Run "make" to regenerate code after modifying this file Config *NodeConfig `json:"config,omitempty"` NumInterfaces int `json:"num-interfaces,omitempty"` Constraints map[string]string `json:"constraints,omitempty"` diff --git a/config/crd/bases/kne.srlinux.dev_srlinuxes.yaml b/config/crd/bases/kne.srlinux.dev_srlinuxes.yaml index 6bf9ac0..aa34def 100644 --- a/config/crd/bases/kne.srlinux.dev_srlinuxes.yaml +++ b/config/crd/bases/kne.srlinux.dev_srlinuxes.yaml @@ -103,10 +103,14 @@ spec: type: string type: object model: + description: Model encodes SR Linux variant (ixr-d3, ixr-6e, etc) type: string num-interfaces: type: integer version: + description: Version may be set in kne topology as a mean to explicitly + provide version information in case it is not encoded in the image + tag type: string type: object status: From 2a844f7e56aed9eb9796fb8001985b1bdfe111df Mon Sep 17 00:00:00 2001 From: Roman Dodin Date: Sat, 30 Jul 2022 22:31:48 +0200 Subject: [PATCH 12/18] added echo --- manifests/variants/kne-entrypoint.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/manifests/variants/kne-entrypoint.yml b/manifests/variants/kne-entrypoint.yml index 16276b9..907cbd6 100644 --- a/manifests/variants/kne-entrypoint.yml +++ b/manifests/variants/kne-entrypoint.yml @@ -10,4 +10,5 @@ data: # this entrypoint ensures that we call topomac script before executing the main entrypoint sudo bash /tmp/topomac/topomac.sh + echo "topomac.sh" script finished exec /entrypoint.sh "$@" From ebfe6c8bf764fe1cb5d1688f14db584fdc4a08cd Mon Sep 17 00:00:00 2001 From: Roman Dodin Date: Sun, 31 Jul 2022 10:39:20 +0200 Subject: [PATCH 13/18] excluded blacklisted linters from testing test files --- .github/workflows/linters/.golangci.yml | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/.github/workflows/linters/.golangci.yml b/.github/workflows/linters/.golangci.yml index 8646bab..f7f49bd 100644 --- a/.github/workflows/linters/.golangci.yml +++ b/.github/workflows/linters/.golangci.yml @@ -65,12 +65,13 @@ linters-settings: issues: # https://github.com/golangci/golangci-lint/issues/2439#issuecomment-1002912465 - exclude-use-default: false + # exclude-use-default: false exclude-rules: - path: _test\.go linters: - - gomnd - - dupl - - structcheck - - unused - - unparam + # - gomnd + # - dupl + # - structcheck + # - unused + # - unparam + - funlen From d70aa3b5d8f078678fef144cbed750fff4cedd8e Mon Sep 17 00:00:00 2001 From: Roman Dodin Date: Sun, 31 Jul 2022 11:45:14 +0200 Subject: [PATCH 14/18] added a note on loading images to kind --- README.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/README.md b/README.md index 175bec8..afd8323 100644 --- a/README.md +++ b/README.md @@ -71,6 +71,16 @@ service-r2 LoadBalancer 10.96.34.36 172.19.0.51 443:30010/TCP,22:30011 To connect with SSH to the `r1` node, use `ssh admin@172.19.0.50` command. +### Loading images to kind cluster + +[Public SR Linux container image](https://github.com/nokia/srlinux-container-image) will be pulled by kind automatically, if Internet access is present. Images which are not available publicy can be uploaded to kind manually: + +```bash +# default kne kind cluster name is `kne` +# which is the last argument in the command +kind load docker-image srlinux:0.0.0-38566 --name kne +``` + ## Controller operations The controller is designed to manage the `Srlinux` custom resource defined with [the following CRD](https://doc.crds.dev/github.com/srl-labs/srl-controller). From 9c05fffab14403342c8a0ab88d524c8714d051c2 Mon Sep 17 00:00:00 2001 From: Roman Dodin Date: Sun, 31 Jul 2022 12:04:37 +0200 Subject: [PATCH 15/18] added license provisioning --- api/types/v1alpha1/srl_version.go | 39 +++++ api/types/v1alpha1/srl_version_test.go | 98 ++++++++++++ api/types/v1alpha1/srlinux_types.go | 62 +++++++- api/types/v1alpha1/srlinux_types_test.go | 141 +++++++++++++++++- api/types/v1alpha1/zz_generated.deepcopy.go | 54 +++++-- .../crd/bases/kne.srlinux.dev_srlinuxes.yaml | 18 +++ controllers/cfgmap.go | 23 ++- controllers/pod.go | 53 ++++++- controllers/secret.go | 68 +++++++++ controllers/srlinux_controller.go | 48 ++++-- hack/boilerplate.go.txt | 34 +++-- 11 files changed, 588 insertions(+), 50 deletions(-) create mode 100644 api/types/v1alpha1/srl_version.go create mode 100644 api/types/v1alpha1/srl_version_test.go create mode 100644 controllers/secret.go diff --git a/api/types/v1alpha1/srl_version.go b/api/types/v1alpha1/srl_version.go new file mode 100644 index 0000000..b766c18 --- /dev/null +++ b/api/types/v1alpha1/srl_version.go @@ -0,0 +1,39 @@ +package v1alpha1 + +import ( + "errors" + "regexp" + "strings" +) + +// ErrVersionParse is an error which is raised when srlinux version is failed to parse. +var ErrVersionParse = errors.New("version parsing failed") + +// SrlVersion represents an sr linux version as a set of fields. +type SrlVersion struct { + Major string `json:"major,omitempty"` + Minor string `json:"minor,omitempty"` + Patch string `json:"patch,omitempty"` + Build string `json:"build,omitempty"` + Commit string `json:"commit,omitempty"` +} + +func parseVersionString(s string) (*SrlVersion, error) { + // for latest or missing tag we consider the version to be an engineering build + // with major = 0 + if strings.ToLower(s) == "latest" || s == "" { + return &SrlVersion{"0", "", "", "", ""}, nil + } + + // https://regex101.com/r/eWS6Ms/1 + re := regexp.MustCompile( + `^v?(?P\d{1,3})\.(?P\d{1,2})\.?(?P\d{1,2})?-?(?P\d{1,10})?-?(?P\S+)?`, + ) + + v := re.FindStringSubmatch(s) + if v == nil { + return nil, ErrVersionParse + } + + return &SrlVersion{v[1], v[2], v[3], v[4], v[5]}, nil +} diff --git a/api/types/v1alpha1/srl_version_test.go b/api/types/v1alpha1/srl_version_test.go new file mode 100644 index 0000000..92c76f2 --- /dev/null +++ b/api/types/v1alpha1/srl_version_test.go @@ -0,0 +1,98 @@ +package v1alpha1 + +import ( + "errors" + "testing" + + "github.com/google/go-cmp/cmp" +) + +func TestParseVersionString(t *testing.T) { + tests := []struct { + desc string + got string + want *SrlVersion + err error + }{ + { + desc: "maj, minor, patch", + got: "21.6.4", + want: &SrlVersion{"21", "6", "4", "", ""}, + }, + { + desc: "maj, minor", + got: "21.6", + want: &SrlVersion{"21", "6", "", "", ""}, + }, + { + desc: "maj, minor and extra string", + got: "21.6-test", + want: &SrlVersion{"21", "6", "", "", "test"}, + }, + { + desc: "maj, minor, patch and extra string", + got: "21.6.11-test", + want: &SrlVersion{"21", "6", "11", "", "test"}, + }, + { + desc: "maj, minor, patch, build and extra string", + got: "21.6.11-235-test", + want: &SrlVersion{"21", "6", "11", "235", "test"}, + }, + { + desc: "maj, minor, patch and build", + got: "21.6.11-235", + want: &SrlVersion{"21", "6", "11", "235", ""}, + }, + { + desc: "0.0", + got: "0.0", + want: &SrlVersion{"0", "0", "", "", ""}, + }, + { + desc: "0.0.0", + got: "0.0.0", + want: &SrlVersion{"0", "0", "0", "", ""}, + }, + { + desc: "0.0.0-34652", + got: "0.0.0-34652", + want: &SrlVersion{"0", "0", "0", "34652", ""}, + }, + { + desc: "latest", + got: "latest", + want: &SrlVersion{"0", "", "", "", ""}, + }, + { + desc: "empty", + got: "", + want: &SrlVersion{"0", "", "", "", ""}, + }, + { + desc: "invalid1", + got: "abcd", + want: nil, + err: ErrVersionParse, + }, + } + + for _, tt := range tests { + t.Run(tt.desc, func(t *testing.T) { + ver, err := parseVersionString(tt.got) + if !errors.Is(err, tt.err) { + t.Fatalf("got error '%v' but expected '%v'", err, tt.err) + } + + if !cmp.Equal(ver, tt.want) { + t.Fatalf( + "%s: actual and expected inputs do not match\nactual: %+v\nexpected:%+v", + tt.desc, + ver, + tt.want, + ) + } + }, + ) + } +} diff --git a/api/types/v1alpha1/srlinux_types.go b/api/types/v1alpha1/srlinux_types.go index 549c6e7..5247bba 100644 --- a/api/types/v1alpha1/srlinux_types.go +++ b/api/types/v1alpha1/srlinux_types.go @@ -17,9 +17,17 @@ limitations under the License. package v1alpha1 import ( + "context" + "fmt" + "strings" + + corev1 "k8s.io/api/core/v1" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" ) +// key value for a combined license file stored in a secret. +const allSecretKey = "all.key" + // SrlinuxSpec defines the desired state of Srlinux. type SrlinuxSpec struct { Config *NodeConfig `json:"config,omitempty"` @@ -50,9 +58,14 @@ type Srlinux struct { Spec SrlinuxSpec `json:"spec,omitempty"` Status SrlinuxStatus `json:"status,omitempty"` + + // parsed NOS version + NOSVersion *SrlVersion `json:"nos-version,omitempty"` + // license key from license secret that contains a license file for this Srlinux + LicenseKey string `json:"license_key,omitempty"` } -//+kubebuilder:object:root=true +// +kubebuilder:object:root=true // SrlinuxList contains a list of Srlinux. type SrlinuxList struct { @@ -112,3 +125,50 @@ func (s *SrlinuxSpec) GetImage() string { return img } + +// GetImageVersion finds an srlinux image version by looking at the Image field of the spec +// as well as at Version field. +// When Version field is set it is returned. +// In other cases, Image string is evaluated and it's tag substring is parsed. +// If no tag is present, or tag is latest, the 0.0 version is assumed to be in use. +func (s *SrlinuxSpec) GetImageVersion() (*SrlVersion, error) { + if s.Version != "" { + return parseVersionString(s.Version) + } + + var tag string + + split := strings.Split(s.GetImage(), ":") + if len(split) == 2 { // nolint: gomnd + tag = split[1] + } + + return parseVersionString(tag) +} + +// InitLicenseKey sets the Srlinux.LicenseKey to a value of a key +// that matches MAJOR-MINOR.key of a passed secret. +// Where MAJOR-MINOR is retrieved from the image version. +// If such key doesn't exist, it checks if a wildcard `all.key` is found in that secret, +// if nothing found, LicenseKey stays empty, which denotes that no license was found for Srlinux. +func (s *Srlinux) InitLicenseKey( + _ context.Context, + secret *corev1.Secret, +) { + if secret == nil { + return + } + + versionSecretKey := fmt.Sprintf("%s-%s.key", s.NOSVersion.Major, s.NOSVersion.Minor) + if _, ok := secret.Data[versionSecretKey]; ok { + s.LicenseKey = versionSecretKey + + return + } + + if _, ok := secret.Data[allSecretKey]; ok { + s.LicenseKey = allSecretKey + + return + } +} diff --git a/api/types/v1alpha1/srlinux_types_test.go b/api/types/v1alpha1/srlinux_types_test.go index afdebca..5f91df5 100644 --- a/api/types/v1alpha1/srlinux_types_test.go +++ b/api/types/v1alpha1/srlinux_types_test.go @@ -1,13 +1,15 @@ package v1alpha1 import ( + "context" + "errors" "testing" "github.com/google/go-cmp/cmp" + corev1 "k8s.io/api/core/v1" ) func TestGetImage(t *testing.T) { - tests := []struct { desc string spec *SrlinuxSpec @@ -34,6 +36,24 @@ func TestGetImage(t *testing.T) { }, want: defaultSRLinuxImageName + ":21.11.1", }, + { + desc: "image without tag", + spec: &SrlinuxSpec{ + Config: &NodeConfig{ + Image: "ghcr.io/nokia/srlinux", + }, + }, + want: "ghcr.io/nokia/srlinux", + }, + { + desc: "image with latest tag", + spec: &SrlinuxSpec{ + Config: &NodeConfig{ + Image: "ghcr.io/nokia/srlinux:latest", + }, + }, + want: "ghcr.io/nokia/srlinux:latest", + }, } for _, tt := range tests { @@ -51,5 +71,124 @@ func TestGetImage(t *testing.T) { }, ) } +} + +func TestGetImageVersion(t *testing.T) { + tests := []struct { + desc string + spec *SrlinuxSpec + want *SrlVersion + err error + }{ + { + desc: "valid version is present", + spec: &SrlinuxSpec{ + Version: "21.11.1", + Config: &NodeConfig{Image: "ghcr.io/nokia/srlinux:somever"}, + }, + want: &SrlVersion{"21", "11", "1", "", ""}, + }, + { + desc: "invalid version is present", + spec: &SrlinuxSpec{ + Version: "abc21.11.1", + Config: &NodeConfig{Image: "ghcr.io/nokia/srlinux:somever"}, + }, + err: ErrVersionParse, + }, + { + desc: "version is not present, valid image tag is given", + spec: &SrlinuxSpec{ + Config: &NodeConfig{Image: "ghcr.io/nokia/srlinux:21.11.1"}, + }, + want: &SrlVersion{"21", "11", "1", "", ""}, + }, + { + desc: "version is not present, invalid image tag is given", + spec: &SrlinuxSpec{ + Config: &NodeConfig{Image: "ghcr.io/nokia/srlinux:21"}, + }, + err: ErrVersionParse, + }, + } + + for _, tt := range tests { + t.Run(tt.desc, func(t *testing.T) { + v, err := tt.spec.GetImageVersion() + + if !errors.Is(err, tt.err) { + t.Fatalf("got error '%v' but expected '%v'", err, tt.err) + } + + if !cmp.Equal(v, tt.want) { + t.Fatalf( + "%s: actual and expected inputs do not match\nactual: %+v\nexpected:%+v", + tt.desc, + v, + tt.want, + ) + } + }, + ) + } +} +func TestInitVersion(t *testing.T) { + tests := []struct { + desc string + srl *Srlinux + secret *corev1.Secret + want string + }{ + { + desc: "secret key matches srl version", + srl: &Srlinux{ + NOSVersion: &SrlVersion{"22", "3", "", "", ""}, + }, + secret: &corev1.Secret{ + Data: map[string][]byte{ + "22-3.key": nil, + "all.key": nil, + }, + }, + want: "22-3.key", + }, + { + desc: "wildcard secret key matches srl version", + srl: &Srlinux{ + NOSVersion: &SrlVersion{"22", "3", "", "", ""}, + }, + secret: &corev1.Secret{ + Data: map[string][]byte{ + "22-6.key": nil, + "all.key": nil, + }, + }, + want: "all.key", + }, + { + desc: "secret does not exist", + srl: &Srlinux{ + NOSVersion: &SrlVersion{"22", "3", "", "", ""}, + }, + secret: nil, + want: "", + }, + } + + for _, tt := range tests { + t.Run(tt.desc, func(t *testing.T) { + tt.srl.InitLicenseKey(context.TODO(), tt.secret) + + if !cmp.Equal(tt.srl.LicenseKey, tt.want) { + t.Fatalf( + "%s: actual and expected inputs do not match\nactual: %+v\nexpected:%+v", + tt.desc, + tt.srl.LicenseKey, + tt.want, + ) + } + }, + ) + } } diff --git a/api/types/v1alpha1/zz_generated.deepcopy.go b/api/types/v1alpha1/zz_generated.deepcopy.go index 83759bb..5b0c633 100644 --- a/api/types/v1alpha1/zz_generated.deepcopy.go +++ b/api/types/v1alpha1/zz_generated.deepcopy.go @@ -2,19 +2,33 @@ // +build !ignore_autogenerated /* -Copyright 2021. +Copyright (c) 2021 Nokia. All rights reserved. -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - http://www.apache.org/licenses/LICENSE-2.0 +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. +1. Redistributions of source code must retain the above copyright notice, this + list of conditions and the following disclaimer. + +2. Redistributions in binary form must reproduce the above copyright notice, + this list of conditions and the following disclaimer in the documentation + and/or other materials provided with the distribution. + +3. Neither the name of the copyright holder nor the names of its + contributors may be used to endorse or promote products derived from + this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR +SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER +CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ // Code generated by controller-gen. DO NOT EDIT. @@ -77,6 +91,21 @@ func (in *NodeConfig) DeepCopy() *NodeConfig { return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *SrlVersion) DeepCopyInto(out *SrlVersion) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SrlVersion. +func (in *SrlVersion) DeepCopy() *SrlVersion { + if in == nil { + return nil + } + out := new(SrlVersion) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *Srlinux) DeepCopyInto(out *Srlinux) { *out = *in @@ -84,6 +113,11 @@ func (in *Srlinux) DeepCopyInto(out *Srlinux) { in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) in.Spec.DeepCopyInto(&out.Spec) out.Status = in.Status + if in.NOSVersion != nil { + in, out := &in.NOSVersion, &out.NOSVersion + *out = new(SrlVersion) + **out = **in + } } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Srlinux. diff --git a/config/crd/bases/kne.srlinux.dev_srlinuxes.yaml b/config/crd/bases/kne.srlinux.dev_srlinuxes.yaml index aa34def..ae9a4be 100644 --- a/config/crd/bases/kne.srlinux.dev_srlinuxes.yaml +++ b/config/crd/bases/kne.srlinux.dev_srlinuxes.yaml @@ -38,8 +38,26 @@ spec: object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string + license_key: + description: license key from license secret that contains a license file + for this Srlinux + type: string metadata: type: object + nos-version: + description: parsed NOS version + properties: + build: + type: string + commit: + type: string + major: + type: string + minor: + type: string + patch: + type: string + type: object spec: description: SrlinuxSpec defines the desired state of Srlinux. properties: diff --git a/controllers/cfgmap.go b/controllers/cfgmap.go index 2a6930c..9820717 100644 --- a/controllers/cfgmap.go +++ b/controllers/cfgmap.go @@ -4,6 +4,7 @@ import ( "context" "github.com/go-logr/logr" + typesv1alpha1 "github.com/srl-labs/srl-controller/api/types/v1alpha1" corev1 "k8s.io/api/core/v1" "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/runtime" @@ -12,24 +13,26 @@ import ( clientgoscheme "k8s.io/client-go/kubernetes/scheme" ) +const srlLicenseSecretName = "srlinux-licenses" + // createConfigMaps creates srlinux-variants and srlinux-topomac config maps which every srlinux pod needs to mount. func createConfigMaps( ctx context.Context, r *SrlinuxReconciler, - ns string, + s *typesv1alpha1.Srlinux, log logr.Logger, ) error { - err := createVariantsCfgMap(ctx, r, ns, log) + err := createVariantsCfgMap(ctx, r, s.Namespace, log) if err != nil { return err } - err = createTopomacScriptCfgMap(ctx, r, ns, log) + err = createTopomacScriptCfgMap(ctx, r, s.Namespace, log) if err != nil { return err } - err = createKNEEntrypointCfgMap(ctx, r, ns, log) + err = createKNEEntrypointCfgMap(ctx, r, s.Namespace, log) return err } @@ -45,7 +48,7 @@ func createVariantsCfgMap( err := r.Get(ctx, types.NamespacedName{Name: variantsCfgMapName, Namespace: ns}, cfgMap) if err != nil && errors.IsNotFound(err) { - log.Info("Creating a new variants configmap") + log.Info("creating a new variants configmap") data, err := VariantsFS.ReadFile("manifests/variants/srl_variants.yml") if err != nil { @@ -65,6 +68,8 @@ func createVariantsCfgMap( if err != nil { return err } + + return nil } return err @@ -81,7 +86,7 @@ func createTopomacScriptCfgMap( err := r.Get(ctx, types.NamespacedName{Name: topomacCfgMapName, Namespace: ns}, cfgMap) if err != nil && errors.IsNotFound(err) { - log.Info("Creating a new topomac script configmap") + log.Info("creating a new topomac script configmap") data, err := VariantsFS.ReadFile("manifests/variants/topomac.yml") if err != nil { @@ -101,6 +106,8 @@ func createTopomacScriptCfgMap( if err != nil { return err } + + return nil } return err @@ -117,7 +124,7 @@ func createKNEEntrypointCfgMap( err := r.Get(ctx, types.NamespacedName{Name: entrypointCfgMapName, Namespace: ns}, cfgMap) if err != nil && errors.IsNotFound(err) { - log.Info("Creating a new kne-entrypoint configmap") + log.Info("creating a new kne-entrypoint configmap") data, err := VariantsFS.ReadFile("manifests/variants/kne-entrypoint.yml") if err != nil { @@ -137,6 +144,8 @@ func createKNEEntrypointCfgMap( if err != nil { return err } + + return nil } return err diff --git a/controllers/pod.go b/controllers/pod.go index ec6390e..3a257df 100644 --- a/controllers/pod.go +++ b/controllers/pod.go @@ -14,7 +14,13 @@ import ( "sigs.k8s.io/controller-runtime/pkg/log" ) -const terminationGracePeriodSeconds = 0 +const ( + terminationGracePeriodSeconds = 0 + licensesVolName = "license" + licenseFileName = "license.key" + licenseMntPath = "/opt/srlinux/etc/license.key" + licenseMntSubPath = "license.key" +) // podForSrlinux returns a srlinux Pod object. func (r *SrlinuxReconciler) podForSrlinux( @@ -85,7 +91,7 @@ func createContainers(s *typesv1a1.Srlinux) []corev1.Container { Privileged: pointer.Bool(true), RunAsUser: pointer.Int64(0), }, - VolumeMounts: createVolumeMounts(), + VolumeMounts: createVolumeMounts(s), }} } @@ -112,7 +118,7 @@ func createAffinity(s *typesv1a1.Srlinux) *corev1.Affinity { } func createVolumes(s *typesv1a1.Srlinux) []corev1.Volume { - return []corev1.Volume{ + vols := []corev1.Volume{ { Name: variantsVolName, VolumeSource: corev1.VolumeSource{ @@ -151,6 +157,12 @@ func createVolumes(s *typesv1a1.Srlinux) []corev1.Volume { }, }, } + + if s.LicenseKey != "" { + vols = append(vols, createLicenseVolume(s)) + } + + return vols } // handleStartupConfig creates volume mounts and volumes for srlinux pod @@ -197,8 +209,8 @@ func handleStartupConfig(s *typesv1a1.Srlinux, pod *corev1.Pod, log logr.Logger) } } -func createVolumeMounts() []corev1.VolumeMount { - return []corev1.VolumeMount{ +func createVolumeMounts(s *typesv1a1.Srlinux) []corev1.VolumeMount { + vms := []corev1.VolumeMount{ { Name: variantsVolName, MountPath: variantsVolMntPath, @@ -213,4 +225,35 @@ func createVolumeMounts() []corev1.VolumeMount { SubPath: entrypointVolMntSubPath, }, } + + if s.LicenseKey != "" { + vms = append(vms, createLicenseVolumeMount()) + } + + return vms +} + +func createLicenseVolume(s *typesv1a1.Srlinux) corev1.Volume { + return corev1.Volume{ + Name: licensesVolName, + VolumeSource: corev1.VolumeSource{ + Secret: &corev1.SecretVolumeSource{ + SecretName: srlLicenseSecretName, + Items: []corev1.KeyToPath{ + { + Key: s.LicenseKey, + Path: licenseFileName, + }, + }, + }, + }, + } +} + +func createLicenseVolumeMount() corev1.VolumeMount { + return corev1.VolumeMount{ + Name: licensesVolName, + MountPath: licenseMntPath, + SubPath: licenseMntSubPath, + } } diff --git a/controllers/secret.go b/controllers/secret.go new file mode 100644 index 0000000..8b13b56 --- /dev/null +++ b/controllers/secret.go @@ -0,0 +1,68 @@ +package controllers + +import ( + "context" + + "github.com/go-logr/logr" + typesv1alpha1 "github.com/srl-labs/srl-controller/api/types/v1alpha1" + corev1 "k8s.io/api/core/v1" + "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/types" +) + +// createSecrets creates secrets such as srlinux-licenses. +func (r *SrlinuxReconciler) createSecrets( + ctx context.Context, + s *typesv1alpha1.Srlinux, + log logr.Logger, +) error { + secret, err := r.copyLicenseSecret(ctx, s, log) + if err != nil { + return err + } + + // set license key matching image version + s.InitLicenseKey(ctx, secret) + + return err +} + +// copyLicenseSecret finds a secret with srlinux licenses in srlinux-controller namespace +// and copies it to the srlinux CR namespace and returns the pointer to the newly created secret. +// If original secret doesn't exist (i.e. when no licenses were provisioned by a user) +// then nothing gets copied and nil returned. +func (r *SrlinuxReconciler) copyLicenseSecret( + ctx context.Context, + s *typesv1alpha1.Srlinux, + log logr.Logger, +) (*corev1.Secret, error) { + secret := &corev1.Secret{} + + // find license secret in controller' ns to copy from + // return silently if not found. + err := r.Get(ctx, types.NamespacedName{Name: srlLicenseSecretName, Namespace: controllerNamespace}, secret) + if err != nil && errors.IsNotFound(err) { + log.Info("secret with licenses is not found in controller's namespace, skipping copy to lab namespace", + "secret name", srlLicenseSecretName, + "controller namespace", controllerNamespace) + + return nil, nil + } + + // copy secret obj from controller's ns to a new secret + // that we put in the lab's ns + newSecret := secret.DeepCopy() + newSecret.Namespace = s.Namespace + newSecret.ResourceVersion = "" + + log.Info("creating secret", + "secret name", srlLicenseSecretName, + "namespace", s.Namespace) + + err = r.Create(ctx, newSecret) + if err != nil { + return nil, err + } + + return newSecret, err +} diff --git a/controllers/srlinux_controller.go b/controllers/srlinux_controller.go index 6162174..defa410 100644 --- a/controllers/srlinux_controller.go +++ b/controllers/srlinux_controller.go @@ -35,6 +35,8 @@ import ( ) const ( + controllerNamespace = "srlinux-controller" + initContainerName = "networkop/init-wait:latest" variantsVolName = "variants" variantsVolMntPath = "/tmp/topo" @@ -45,9 +47,11 @@ const ( topomacVolMntPath = "/tmp/topomac" topomacCfgMapName = "srlinux-topomac-script" - entrypointVolName = "kne-entrypoint" - entrypointVolMntPath = "/kne-entrypoint.sh" - entrypointVolMntSubPath = "kne-entrypoint.sh" // used to enable setting file permissions for a file + entrypointVolName = "kne-entrypoint" + entrypointVolMntPath = "/kne-entrypoint.sh" + // used to enable mounting a file in an existing folder + // https://stackoverflow.com/questions/33415913/whats-the-best-way-to-share-mount-one-file-into-a-pod + entrypointVolMntSubPath = "kne-entrypoint.sh" entrypointCfgMapName = "srlinux-kne-entrypoint" // default path to a startup config file @@ -89,6 +93,7 @@ func (r *SrlinuxReconciler) Reconcile( req ctrl.Request, ) (ctrl.Result, error) { log := log.FromContext(ctx) + srlinux := &typesv1alpha1.Srlinux{} if res, isReturn, err := r.checkSrlinuxCR(ctx, log, req, srlinux); isReturn { @@ -135,7 +140,7 @@ func (r *SrlinuxReconciler) checkSrlinuxCR( return ctrl.Result{}, true, nil } // Error reading the object - requeue the request. - log.Error(err, "Failed to get Srlinux") + log.Error(err, "failed to get Srlinux") return ctrl.Result{}, true, err } @@ -149,25 +154,36 @@ func (r *SrlinuxReconciler) checkSrlinuxPod( srlinux *typesv1alpha1.Srlinux, found *corev1.Pod, ) (ctrl.Result, bool, error) { - err := r.Get(ctx, types.NamespacedName{Name: srlinux.Name, Namespace: srlinux.Namespace}, found) + // get NOS version + v, err := srlinux.Spec.GetImageVersion() + if err != nil { + return ctrl.Result{}, true, err + } + + log.Info("SR Linux image version parsed", "version", v) + + srlinux.NOSVersion = v + + err = r.Get(ctx, types.NamespacedName{Name: srlinux.Name, Namespace: srlinux.Namespace}, found) if err != nil && errors.IsNotFound(err) { - err = createConfigMaps(ctx, r, srlinux.Namespace, log) + err = createConfigMaps(ctx, r, srlinux, log) + if err != nil { + return ctrl.Result{}, true, err + } + + err = r.createSecrets(ctx, srlinux, log) if err != nil { return ctrl.Result{}, true, err } + // Define a new srlinux pod pod := r.podForSrlinux(ctx, srlinux) - log.Info("Creating a new Pod", "Pod.Namespace", pod.Namespace, "Pod.Name", pod.Name) + log.Info("creating a new pod") err = r.Create(ctx, pod) if err != nil { log.Error( - err, - "Failed to create new Pod", - "Pod.Namespace", - pod.Namespace, - "Pod.Name", - pod.Name, + err, "failed to create new Pod", ) return ctrl.Result{}, true, err @@ -176,7 +192,7 @@ func (r *SrlinuxReconciler) checkSrlinuxPod( // Pod created successfully - return and requeue return ctrl.Result{Requeue: true}, true, nil } else if err != nil { - log.Error(err, "Failed to get Pod") + log.Error(err, "failed to get Pod") return ctrl.Result{}, true, err } @@ -191,12 +207,12 @@ func (r *SrlinuxReconciler) updateSrlinuxStatus( found *corev1.Pod, ) (ctrl.Result, bool, error) { if !reflect.DeepEqual(found.Spec.Containers[0].Image, srlinux.Status.Image) { - log.Info("Updating srlinux image status to", "image", found.Spec.Containers[0].Image) + log.Info("updating srlinux image status to", "image", found.Spec.Containers[0].Image) srlinux.Status.Image = found.Spec.Containers[0].Image err := r.Status().Update(ctx, srlinux) if err != nil { - log.Error(err, "Failed to update Srlinux status") + log.Error(err, "failed to update Srlinux status") return ctrl.Result{}, true, err } diff --git a/hack/boilerplate.go.txt b/hack/boilerplate.go.txt index 45dbbbb..c975399 100644 --- a/hack/boilerplate.go.txt +++ b/hack/boilerplate.go.txt @@ -1,15 +1,29 @@ /* -Copyright 2021. +Copyright (c) 2021 Nokia. All rights reserved. -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - http://www.apache.org/licenses/LICENSE-2.0 +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. +1. Redistributions of source code must retain the above copyright notice, this + list of conditions and the following disclaimer. + +2. Redistributions in binary form must reproduce the above copyright notice, + this list of conditions and the following disclaimer in the documentation + and/or other materials provided with the distribution. + +3. Neither the name of the copyright holder nor the names of its + contributors may be used to endorse or promote products derived from + this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR +SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER +CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ \ No newline at end of file From 101ba4302332be0e3bb955f125897dc9d3f6c6f5 Mon Sep 17 00:00:00 2001 From: Roman Dodin Date: Sun, 31 Jul 2022 12:15:28 +0200 Subject: [PATCH 16/18] uncuddle --- controllers/srlinux_controller.go | 1 + 1 file changed, 1 insertion(+) diff --git a/controllers/srlinux_controller.go b/controllers/srlinux_controller.go index defa410..f26ee3a 100644 --- a/controllers/srlinux_controller.go +++ b/controllers/srlinux_controller.go @@ -178,6 +178,7 @@ func (r *SrlinuxReconciler) checkSrlinuxPod( // Define a new srlinux pod pod := r.podForSrlinux(ctx, srlinux) + log.Info("creating a new pod") err = r.Create(ctx, pod) From d20c6115f80f68bed4f6a44c187c141c66180b0c Mon Sep 17 00:00:00 2001 From: Roman Dodin Date: Sun, 31 Jul 2022 21:48:08 +0200 Subject: [PATCH 17/18] added license readme --- README.md | 8 ++++++- docs/using-licenses.md | 49 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 56 insertions(+), 1 deletion(-) create mode 100644 docs/using-licenses.md diff --git a/README.md b/README.md index afd8323..4d4a6e4 100644 --- a/README.md +++ b/README.md @@ -81,6 +81,12 @@ To connect with SSH to the `r1` node, use `ssh admin@172.19.0.50` command. kind load docker-image srlinux:0.0.0-38566 --name kne ``` +## Using license files + +To remove the packets-per-second limit of a public container image or to launch chassis-based variants of SR Linux (ixr-6e/10e) KNE users should provide a valid license file to the `srl-controller`. + +Navigate to [Using license files](docs/using-licenses.md) document to have a detailed explanation on that topic. + ## Controller operations The controller is designed to manage the `Srlinux` custom resource defined with [the following CRD](https://doc.crds.dev/github.com/srl-labs/srl-controller). @@ -104,7 +110,7 @@ When a deletion happens on `Srlinux` resource, the reconcile loop does nothing. This repo contains a clientset for API access to the `Srlinux` custom resource. Check [kne repo](https://github.com/openconfig/kne/blob/fc195a73035bcbf344791979ca3e067be47a249c/topo/node/srl/srl.go#L46) to see how this can be done. -## Building srl-controller container image +## Building `srl-controller` container image To build `srl-controller` container image execute: diff --git a/docs/using-licenses.md b/docs/using-licenses.md new file mode 100644 index 0000000..f257984 --- /dev/null +++ b/docs/using-licenses.md @@ -0,0 +1,49 @@ +# Using license files + +To remove the packets-per-second limit of a public container image or to launch chassis-based variants of SR Linux (ixr-6e/10e) KNE users should provide a valid license file to the `srl-controller`. + +License file provisioning is handled by the `srl-controller`. Users should create a k8s secret with license text blobs stored as keys for a controller to pick it up and use with SR Linux pods. In the next sections, we provide examples for the complete workflow of license provisioning. + +## Creating a secret with licenses + +SR Linux license file can contain a single license blob for a certain release or pack multiple license blobs for several versions. SR Linux NOS can conveniently find a matching license in a file that contains several licenses automatically. + +A license file with multiple licenses can look similar to that: + +```text +# +# srl +# +00000000-0000-0000-0000-000000000000 aACUAsYXC0NTA1NERLABiSBETTERtHAnKNEAAAAA # srl_rel_22_03_* +00000000-0000-0000-0000-000000000000 aACUAsYXC0NTA1NERLABiSBETTERtHAnKNEAAAAA # srl_rel_22_06_* +00000000-0000-0000-0000-000000000000 aACUAsYXC0NTA1NERLABiSBETTERtHAnKNEAAAAA # srl_rel_22_11_* +``` + +A file like that contains licenses for SR Linux releases `22.3.*`, `22.6.*`, `22.11.*`, but, for instance, not for `21.11.*`. + +Because a single license file can contain multiple license blob, users can maintain a single file and append license blobs to it as the new releases come out. + +For the sake of an argument, let's assume that a license file that contains license blobs is named `licenses.key` and exists in a current working directory. With a license file available, users should create a Secret in the `srlinux-controller` namespace with a license file blob contained under the `all.key` key. + +```bash +kubectl create namespace srlinux-controller; \ +kubectl create -n srlinux-controller \ + secret generic srlinux-licenses --from-file=all.key=licenses.key \ + --dry-run=client --save-config -o yaml | \ + kubectl apply -f - +``` + +> **Note** +> The above snippet ensures that `srlinux-controller` namespace exists, and then creates a Secret object from `licenses.key` file and puts its content under `all.key` key. + +Now you should have a Secret object in `srlinux-controller` namespace that contains SR Linux licenses. + +## License mount + +Once a Secret with license information is created, SR Linux pods will have a new volume mounted by the controller with the contents of the original license file by the path `/opt/srlinux/etc/license.key`. + +SR Linux NOS then will read this file at startup and will use a license if a valid string is found in that file. If no valid license is found the system will boot as if no license file was provided. + +## Updating licenses + +If you wish to add/remove a license to/from your collection of licenses you simply modify the existing file which you used to create a Secret object from and reinvoke the same command. From 8759a6b94b3579985e6c00d9dfa7244a4ce20782 Mon Sep 17 00:00:00 2001 From: Roman Dodin Date: Sun, 31 Jul 2022 21:48:34 +0200 Subject: [PATCH 18/18] prepare 0.4.0 release --- config/manager/kustomization.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/config/manager/kustomization.yaml b/config/manager/kustomization.yaml index f617449..3621a31 100644 --- a/config/manager/kustomization.yaml +++ b/config/manager/kustomization.yaml @@ -13,4 +13,4 @@ kind: Kustomization images: - name: controller newName: ghcr.io/srl-labs/srl-controller - newTag: 0.3.4 + newTag: 0.4.0