diff --git a/README.md b/README.md index 72f70e2..d0e48c5 100644 --- a/README.md +++ b/README.md @@ -27,9 +27,9 @@ This page describe how to deploy Redis Enterprise on Kubernetes using the Redis The following are the images and tags for this release: | Component | k8s | Openshift | | --- | --- | --- | -| Redis Enterprise | `redislabs/redis:6.0.12-57` | `redislabs/redis:6.0.12-57.rhel7-openshift` | -| Operator | `redislabs/operator:6.0.12-5` | `redislabs/operator:6.0.12-5` | -| Services Rigger | `redislabs/k8s-controller:6.0.12-5` | `redislabs/k8s-controller:6.0.12-5` | +| Redis Enterprise | `redislabs/redis:6.0.20-69` | `redislabs/redis:6.0.20-69.rhel7-openshift` | +| Operator | `redislabs/operator:6.0.20-4` | `redislabs/operator:6.0.20-4` | +| Services Rigger | `redislabs/k8s-controller:6.0.20-4` | `redislabs/k8s-controller:6.0.20-4` | > * RedHat certified images are available on [Redhat Catalog](https://access.redhat.com/containers/#/product/71f6d1bb3408bd0d)
@@ -130,7 +130,6 @@ This is the fastest way to get up and running with a new Redis Enterprise on Kub kubectl patch ValidatingWebhookConfiguration redb-admission --patch "$(cat modified-webhook.yaml)" ``` * Verify the installation - In order to verify that the all the components of the Admission Controller are installed correctly, we will try to apply an invalid resource that should force the admission controller to reject it. If it applies succesfully, it means the admission controller has not been hooked up correctly. ```shell script @@ -156,7 +155,7 @@ This is the fastest way to get up and running with a new Redis Enterprise on Kub Create a `RedisEnterpriseDatabase` (REDB) by using Custom Resource. > Note: An example REDB.yaml file may be found [HERE] (https://github.com/RedisLabs/redis-enterprise-k8s-docs/tree/master/examples/v1alpha1). This is an alternative to copying and pasting the example code below into the CLI - + The Redis Enterprise Operator can be instructed to manage databases on the Redis Enterprise Cluster using the REDB custom resource. Example: ```yaml @@ -311,7 +310,7 @@ The operator deploys a `RedisEnterpriseCluster` with default configurations valu redisEnterpriseImageSpec: imagePullPolicy: IfNotPresent repository: redislabs/redis - versionTag: 6.0.12-57 + versionTag: 6.0.20-69 ``` * Persistence @@ -413,21 +412,21 @@ For example: redisEnterpriseImageSpec: imagePullPolicy: IfNotPresent repository: harbor.corp.local/redisenterprise/redis - versionTag: 6.0.12-57 + versionTag: 6.0.20-69 ``` ```yaml redisEnterpriseServicesRiggerImageSpec: imagePullPolicy: IfNotPresent repository: harbor.corp.local/redisenterprise/k8s-controller - versionTag: 6.0.12-5 + versionTag: 6.0.20-4 ``` ```yaml bootstrapperImageSpec: imagePullPolicy: IfNotPresent repository: harbor.corp.local/redisenterprise/operator - versionTag: 6.0.12-5 + versionTag: 6.0.20-4 ``` In Operator Deployment spec (operator.yaml): @@ -439,7 +438,7 @@ spec: spec: containers: - name: redis-enterprise-operator - image: harbor.corp.local/redisenterprise/operator:6.0.12-5 + image: harbor.corp.local/redisenterprise/operator:6.0.20-4 ``` Image specification follow the [K8s Container schema](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.10/#container-v1-core). @@ -490,7 +489,7 @@ spec: The Operator automates and simplifies the upgrade process. The Redis Enterprise Cluster Software, and the Redis Enterprise Operator for Kubernetes versions are tightly coupled and should be upgraded together. It is recommended to use the bundle.yaml to upgrade, as it loads all the relevant CRD documents for this version. If the updated CRDs are not loaded, the operator might fail. -There are two ways to upgrade - either set 'autoUpgradeRedisEnterprise' within the Redis Enterprise Cluster Spec to instruct the operator to automatically upgrade to the compatible version, or specify the correct Redis Enterprise image manually using the versionTag attribute. The Redis Enterprise Version compatible with this release is 6.0.12-57 +There are two ways to upgrade - either set 'autoUpgradeRedisEnterprise' within the Redis Enterprise Cluster Spec to instruct the operator to automatically upgrade to the compatible version, or specify the correct Redis Enterprise image manually using the versionTag attribute. The Redis Enterprise Version compatible with this release is 6.0.20-69 ```yaml autoUpgradeRedisEnterprise: true @@ -499,7 +498,7 @@ There are two ways to upgrade - either set 'autoUpgradeRedisEnterprise' within t Alternatively: ```yaml RedisEnterpriseImageSpec: - versionTag: redislabs/redis:6.0.12-57 + versionTag: redislabs/redis:6.0.20-69 ``` ## Supported K8S Distributions @@ -508,22 +507,20 @@ Supported versions (platforms/versions that are not listed are not supported): | Distribution | Support Status | |---------------------------------|----------------| | Openshift 3.11 (K8s 1.11) | supported | -| Openshift 4.1 (K8s 1.13) | deprecated* | -| Openshift 4.2 (K8s 1.14) | deprecated* | -| Openshift 4.3 (K8s 1.16) | deprecated* | -| Openshift 4.4 (K8s 1.17) | supported | +| Openshift 4.4 (K8s 1.17) | deprecated | | OpenShift 4.5 (K8s 1.18) | supported | | OpenShift 4.6 (K8s 1.19) | supported | -| KOPS vanilla 1.13 | deprecated | -| KOPS vanilla 1.14 | deprecated | -| KOPS vanilla 1.15 | supported | +| OpenShift 4.7 (K8s 1.20) | supported | +| KOPS vanilla 1.15 | deprecated | | KOPS vanilla 1.16 | supported | | KOPS vanilla 1.17 | supported | | KOPS vanilla 1.18 | supported | | KOPS vanilla 1.19 | supported | -| GKE 1.14 | deprecated** | -| GKE 1.15 | supported | -| GKE 1.16 | supported | +| GKE 1.15 | deprecated | +| GKE 1.16 | deprecated | +| GKE 1.17 | supported | +| GKE 1.18 | supported | +| GKE 1.19 | supported | | Rancher 2.4 (K8s 1.17) | supported | | Rancher 2.4 (K8s 1.18) | supported | | Rancher 2.5 (K8s 1.17) | supported | diff --git a/admission.bundle.yaml b/admission.bundle.yaml index a1c1a04..d95cced 100644 --- a/admission.bundle.yaml +++ b/admission.bundle.yaml @@ -57,7 +57,7 @@ spec: serviceAccountName: redis-enterprise-admission containers: - name: admin - image: redislabs/operator:6.0.12-5 + image: redislabs/operator:6.0.20-4 command: - /usr/local/bin/admission imagePullPolicy: Always @@ -84,33 +84,4 @@ spec: path: /healthz port: 8443 scheme: HTTPS - initContainers: - - name: admin-init - image: redislabs/operator:6.0.12-5 - command: - - /usr/local/bin/admission - args: - - '-generate-tls' - imagePullPolicy: Always - env: - - name: POD_NAME - valueFrom: - fieldRef: - fieldPath: metadata.name - - name: POD_NAMESPACE - valueFrom: - fieldRef: - fieldPath: metadata.namespace - - name: POD_IP - valueFrom: - fieldRef: - fieldPath: status.podIP - resources: - limits: - cpu: 2000m - memory: 256Mi - requests: - cpu: 100m - memory: 256Mi - --- diff --git a/admission/README.md b/admission/README.md index 5a7e277..82a67a8 100644 --- a/admission/README.md +++ b/admission/README.md @@ -92,6 +92,7 @@ metadata: name: redis-enterprise-database spec: evictionPolicy: illegal + defaultUser: false EOF ``` diff --git a/admission/deployment.yaml b/admission/deployment.yaml index baae0f8..e6bd915 100644 --- a/admission/deployment.yaml +++ b/admission/deployment.yaml @@ -14,7 +14,7 @@ spec: serviceAccountName: redis-enterprise-admission containers: - name: admin - image: redislabs/operator:6.0.12-5 + image: redislabs/operator:6.0.20-4 command: - /usr/local/bin/admission imagePullPolicy: Always @@ -41,32 +41,3 @@ spec: path: /healthz port: 8443 scheme: HTTPS - initContainers: - - name: admin-init - image: redislabs/operator:6.0.12-5 - command: - - /usr/local/bin/admission - args: - - '-generate-tls' - imagePullPolicy: Always - env: - - name: POD_NAME - valueFrom: - fieldRef: - fieldPath: metadata.name - - name: POD_NAMESPACE - valueFrom: - fieldRef: - fieldPath: metadata.namespace - - name: POD_IP - valueFrom: - fieldRef: - fieldPath: status.podIP - resources: - limits: - cpu: 2000m - memory: 256Mi - requests: - cpu: 100m - memory: 256Mi - diff --git a/bundle.yaml b/bundle.yaml index 2cf5c44..c6d5ccf 100644 --- a/bundle.yaml +++ b/bundle.yaml @@ -21,7 +21,7 @@ rules: verbs: ["get", "list", "watch"] - apiGroups: [""] resources: ["events"] - verbs: ["create"] + verbs: ["create", "patch"] - apiGroups: ["apps"] resources: ["deployments", "statefulsets", "replicasets"] verbs: ["*"] @@ -508,6 +508,20 @@ spec: serviceAccountName: description: Name of the service account to use type: string + redisEnterpriseServicesConfiguration: + description: Cluster optional services settings + properties: + mdnsServer: + properties: + operatingMode: + enum: + - enabled + - disabled + type: string + required: + - operatingMode + type: object + type: object servicesRiggerSpec: description: Specification for service rigger properties: @@ -1797,7 +1811,7 @@ spec: serviceAccountName: redis-enterprise-operator containers: - name: redis-enterprise-operator - image: redislabs/operator:6.0.12-5 + image: redislabs/operator:6.0.20-4 command: - redis-enterprise-operator imagePullPolicy: Always @@ -2151,6 +2165,12 @@ spec: defaultUser: description: Is connecting with a default user allowed? type: boolean + ossCluster: + description: OSS Cluster mode option + type: boolean + proxyPolicy: + description: The policy used for proxy binding to the endpoint + type: string evictionPolicy: description: Database eviction policy. see more https://docs.redislabs.com/latest/rs/administering/database-operations/eviction-policy/ type: string diff --git a/crds/v1/rec_crd.yaml b/crds/v1/rec_crd.yaml index e121417..43ee4fb 100644 --- a/crds/v1/rec_crd.yaml +++ b/crds/v1/rec_crd.yaml @@ -433,6 +433,20 @@ spec: serviceAccountName: description: Name of the service account to use type: string + redisEnterpriseServicesConfiguration: + description: Cluster optional services settings + properties: + mdnsServer: + properties: + operatingMode: + enum: + - enabled + - disabled + type: string + required: + - operatingMode + type: object + type: object servicesRiggerSpec: description: Specification for service rigger properties: diff --git a/crds/v1alpha1/rec_crd.yaml b/crds/v1alpha1/rec_crd.yaml index 9788a7a..5b6da48 100644 --- a/crds/v1alpha1/rec_crd.yaml +++ b/crds/v1alpha1/rec_crd.yaml @@ -135,6 +135,18 @@ spec: value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/' type: object type: object + clusterCredentialSecretName: + description: Secret Name/Path to use for Cluster Credentials. If left + blank, will use cluster name + type: string + clusterCredentialSecretRole: + description: Used only if ClusterCredentialSecretType is vault, to define + vault role to be used. If blank, defaults to "redis-enterprise-operator" + type: string + clusterCredentialSecretType: + description: Type of Secret to use for ClusterCredential, Vault, Kuberetes,... + If left blank, will default ot kubernetes secrets + type: string clusterRecovery: description: ClusterRecovery initiates cluster recovery when set to true. Note that this field is cleared automatically after the cluster @@ -433,6 +445,20 @@ spec: serviceAccountName: description: Name of the service account to use type: string + redisEnterpriseServicesConfiguration: + description: Cluster optional services settings + properties: + mdnsServer: + properties: + operatingMode: + enum: + - enabled + - disabled + type: string + required: + - operatingMode + type: object + type: object servicesRiggerSpec: description: Specification for service rigger properties: @@ -1104,6 +1130,10 @@ spec: username: description: Username for the admin user of Redis Enterprise type: string + vaultCASecret: + description: K8s secret name containing Vault's CA cert - defaults to + "vault-ca-cert" + type: string volumes: description: additional volumes items: diff --git a/crds/v1alpha1/redb_crd.yaml b/crds/v1alpha1/redb_crd.yaml index 6965ad4..69a754b 100644 --- a/crds/v1alpha1/redb_crd.yaml +++ b/crds/v1alpha1/redb_crd.yaml @@ -327,6 +327,12 @@ spec: defaultUser: description: Is connecting with a default user allowed? type: boolean + ossCluster: + description: OSS Cluster mode option + type: boolean + proxyPolicy: + description: The policy used for proxy binding to the endpoint + type: string evictionPolicy: description: Database eviction policy. see more https://docs.redislabs.com/latest/rs/administering/database-operations/eviction-policy/ type: string diff --git a/examples/v1alpha1/redb.yaml b/examples/v1alpha1/redb.yaml index 2ca1b87..83c025d 100644 --- a/examples/v1alpha1/redb.yaml +++ b/examples/v1alpha1/redb.yaml @@ -5,4 +5,3 @@ metadata: spec: redisEnterpriseCluster: name: redis-enterprise - memorySize: 100MB diff --git a/multi-namespace-redb/operator.yaml b/multi-namespace-redb/operator.yaml index b326831..30b3cc4 100644 --- a/multi-namespace-redb/operator.yaml +++ b/multi-namespace-redb/operator.yaml @@ -15,7 +15,7 @@ spec: serviceAccountName: redis-enterprise-operator containers: - name: redis-enterprise-operator - image: redislabs/operator:6.0.12-5 + image: redislabs/operator:6.0.20-4 command: - redis-enterprise-operator imagePullPolicy: Always diff --git a/openshift.bundle.yaml b/openshift.bundle.yaml index fe4bbc4..6de4496 100644 --- a/openshift.bundle.yaml +++ b/openshift.bundle.yaml @@ -37,7 +37,7 @@ rules: verbs: ["get", "list", "watch"] - apiGroups: [""] resources: ["events"] - verbs: ["create"] + verbs: ["create", "patch"] - apiGroups: ["apps"] resources: ["deployments", "statefulsets", "replicasets"] verbs: ["*"] @@ -524,6 +524,20 @@ spec: serviceAccountName: description: Name of the service account to use type: string + redisEnterpriseServicesConfiguration: + description: Cluster optional services settings + properties: + mdnsServer: + properties: + operatingMode: + enum: + - enabled + - disabled + type: string + required: + - operatingMode + type: object + type: object servicesRiggerSpec: description: Specification for service rigger properties: @@ -1813,7 +1827,7 @@ spec: serviceAccount: redis-enterprise-operator containers: - name: redis-enterprise-operator - image: redislabs/operator:6.0.12-5 + image: redislabs/operator:6.0.20-4 securityContext: runAsUser: 1001 command: @@ -1828,6 +1842,8 @@ spec: valueFrom: fieldRef: fieldPath: metadata.name + - name: DEPLOY_RHEL7 + value: "true" - name: DATABASE_CONTROLLER_ENABLED value: "true" resources: @@ -2167,6 +2183,12 @@ spec: defaultUser: description: Is connecting with a default user allowed? type: boolean + ossCluster: + description: OSS Cluster mode option + type: boolean + proxyPolicy: + description: The policy used for proxy binding to the endpoint + type: string evictionPolicy: description: Database eviction policy. see more https://docs.redislabs.com/latest/rs/administering/database-operations/eviction-policy/ type: string diff --git a/openshift/operator_rhel.yaml b/openshift/operator_rhel.yaml index 3eb62cb..40a1442 100644 --- a/openshift/operator_rhel.yaml +++ b/openshift/operator_rhel.yaml @@ -15,7 +15,7 @@ spec: serviceAccount: redis-enterprise-operator containers: - name: redis-enterprise-operator - image: redislabs/operator:6.0.12-5 + image: redislabs/operator:6.0.20-4 securityContext: runAsUser: 1001 command: @@ -30,6 +30,8 @@ spec: valueFrom: fieldRef: fieldPath: metadata.name + - name: DEPLOY_RHEL7 + value: "true" - name: DATABASE_CONTROLLER_ENABLED value: "true" resources: diff --git a/openshift/rec_rhel.yaml b/openshift/rec_rhel.yaml index fa28deb..0add19a 100644 --- a/openshift/rec_rhel.yaml +++ b/openshift/rec_rhel.yaml @@ -7,7 +7,7 @@ spec: nodes: 3 redisEnterpriseImageSpec: repository: registry.connect.redhat.com/redislabs/redis-enterprise - versionTag: 6.0.12-57.rhel7-openshift + versionTag: 6.0.20-69.rhel7-openshift redisEnterpriseServicesRiggerImageSpec: repository: registry.connect.redhat.com/redislabs/services-manager bootstrapperImageSpec: diff --git a/openshift/role.yaml b/openshift/role.yaml index 698941c..f082444 100644 --- a/openshift/role.yaml +++ b/openshift/role.yaml @@ -20,7 +20,7 @@ rules: verbs: ["get", "list", "watch"] - apiGroups: [""] resources: ["events"] - verbs: ["create"] + verbs: ["create", "patch"] - apiGroups: ["apps"] resources: ["deployments", "statefulsets", "replicasets"] verbs: ["*"] diff --git a/operator.yaml b/operator.yaml index fd16e51..f2b6d86 100644 --- a/operator.yaml +++ b/operator.yaml @@ -15,7 +15,7 @@ spec: serviceAccountName: redis-enterprise-operator containers: - name: redis-enterprise-operator - image: redislabs/operator:6.0.12-5 + image: redislabs/operator:6.0.20-4 command: - redis-enterprise-operator imagePullPolicy: Always diff --git a/redis_enterprise_cluster_api.md b/redis_enterprise_cluster_api.md index 3150a00..9447eb8 100644 --- a/redis_enterprise_cluster_api.md +++ b/redis_enterprise_cluster_api.md @@ -4,21 +4,30 @@ This document describes the parameters for the Redis Enterprise Cluster custom r ## Table of Contents * [Objects](#objects) * [ActiveActive](#activeactive) + * [CmServer](#cmserver) + * [CrdbCoordinator](#crdbcoordinator) + * [CrdbWorker](#crdbworker) * [ImageSpec](#imagespec) * [LicenseStatus](#licensestatus) + * [MdnsServer](#mdnsserver) * [Module](#module) + * [PdnsServer](#pdnsserver) * [PersistentConfigurationSpec](#persistentconfigurationspec) * [RedisEnterpriseCluster](#redisenterprisecluster) * [RedisEnterpriseClusterList](#redisenterpriseclusterlist) * [RedisEnterpriseClusterSpec](#redisenterpriseclusterspec) * [RedisEnterpriseClusterStatus](#redisenterpriseclusterstatus) + * [RedisEnterpriseServicesConfiguration](#redisenterpriseservicesconfiguration) + * [Saslauthd](#saslauthd) * [ServicesRiggerConfigurationSpec](#servicesriggerconfigurationspec) * [SlaveHA](#slaveha) + * [StatsArchiver](#statsarchiver) * [UpgradeSpec](#upgradespec) * [Enums](#enums) * [ActiveActiveMethod](#activeactivemethod) * [ClusterEventReason](#clustereventreason) * [ClusterState](#clusterstate) + * [OperatingMode](#operatingmode) * [SpecStatusName](#specstatusname) ## Objects @@ -33,6 +42,30 @@ This document describes the parameters for the Redis Enterprise Cluster custom r | ingressAnnotations | Used for ingress controllers such as ha-proxy or nginx in GKE | map[string]string | | false | [Back to Table of Contents](#table-of-contents) +### CmServer + + +| Field | Description | Scheme | Default Value | Required | +| ----- | ----------- | ------ | -------- | -------- | +| operatingMode | Whether to enable/disable the CM server | [OperatingMode](#operatingmode) | | true | +[Back to Table of Contents](#table-of-contents) + +### CrdbCoordinator + + +| Field | Description | Scheme | Default Value | Required | +| ----- | ----------- | ------ | -------- | -------- | +| operatingMode | Whether to enable/disable the crdb coordinator process | [OperatingMode](#operatingmode) | | true | +[Back to Table of Contents](#table-of-contents) + +### CrdbWorker + + +| Field | Description | Scheme | Default Value | Required | +| ----- | ----------- | ------ | -------- | -------- | +| operatingMode | Whether to enable/disable the crdb worker processes | [OperatingMode](#operatingmode) | | true | +[Back to Table of Contents](#table-of-contents) + ### ImageSpec Image specification @@ -54,6 +87,14 @@ Image specification | shardsLimit | Number of redis shards allowed under this license | int32 | | true | [Back to Table of Contents](#table-of-contents) +### MdnsServer + + +| Field | Description | Scheme | Default Value | Required | +| ----- | ----------- | ------ | -------- | -------- | +| operatingMode | Whether to enable/disable the Multicast DNS server | [OperatingMode](#operatingmode) | | true | +[Back to Table of Contents](#table-of-contents) + ### Module @@ -64,6 +105,14 @@ Image specification | versions | | []string | | true | [Back to Table of Contents](#table-of-contents) +### PdnsServer + + +| Field | Description | Scheme | Default Value | Required | +| ----- | ----------- | ------ | -------- | -------- | +| operatingMode | Whether to enable/disable the pdns server | [OperatingMode](#operatingmode) | | true | +[Back to Table of Contents](#table-of-contents) + ### PersistentConfigurationSpec Specification for Redis Enterprise Cluster persistence @@ -116,7 +165,7 @@ RedisEnterpriseClusterSpec defines the desired state of RedisEnterpriseCluster | pullSecrets | PullSecrets is an optional list of references to secrets in the same namespace to use for pulling any of the images. If specified, these secrets will be passed to individual puller implementations for them to use. More info: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/ | [][v1.LocalObjectReference](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.18/#localobjectreference-v1-core) | empty | false | | persistentSpec | Specification for Redis Enterprise Cluster persistence | [PersistentConfigurationSpec](#persistentconfigurationspec) | | false | | sideContainersSpec | Specification for a side container that will be added to each Redis Enterprise pod | []v1.Container | empty | false | -| extraLabels | Labels that the user defines for their convenience | map[string]string | empty | false | +| extraLabels | Labels that the user defines for their convenience. Note that Persistent Volume Claims would only be labeled with the extra labels specified during the cluster's creation (modifying this field when the cluster is running won't affect the Persistent Volume | map[string]string | empty | false | | podAntiAffinity | Override for the default anti-affinity rules of the Redis Enterprise pods | *v1.PodAntiAffinity | | false | | antiAffinityAdditionalTopologyKeys | Additional antiAffinity terms in order to support installation on different zones/vcenters | []string | | false | | activeActive | Specification for ActiveActive setup | *[ActiveActive](#activeactive) | | false | @@ -131,6 +180,11 @@ RedisEnterpriseClusterSpec defines the desired state of RedisEnterpriseCluster | podAnnotations | pod annotations | map[string]string | | false | | podTolerations | Tolerations that are added to all managed pods. for more information: https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/ | [][v1.Toleration](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.18/#toleration-v1-core) | empty | false | | slaveHA | Slave high availability mechanism configuration. | *[SlaveHA](#slaveha) | | false | +| clusterCredentialSecretName | Secret Name/Path to use for Cluster Credentials. If left blank, will use cluster name | string | | false | +| clusterCredentialSecretType | Type of Secret to use for ClusterCredential, Vault, Kuberetes,... If left blank, will default ot kubernetes secrets | string | | true | +| clusterCredentialSecretRole | Used only if ClusterCredentialSecretType is vault, to define vault role to be used. If blank, defaults to \"redis-enterprise-operator\" | string | | true | +| vaultCASecret | K8s secret name containing Vault's CA cert - defaults to \"vault-ca-cert\" | string | | true | +| redisEnterpriseServicesConfiguration | RS Cluster optional services settings | *[RedisEnterpriseServicesConfiguration](#redisenterpriseservicesconfiguration) | | false | [Back to Table of Contents](#table-of-contents) ### RedisEnterpriseClusterStatus @@ -144,6 +198,28 @@ RedisEnterpriseClusterStatus defines the observed state of RedisEnterpriseCluste | licenseStatus | State of the Cluster's License | *[LicenseStatus](#licensestatus) | | false | [Back to Table of Contents](#table-of-contents) +### RedisEnterpriseServicesConfiguration + + +| Field | Description | Scheme | Default Value | Required | +| ----- | ----------- | ------ | -------- | -------- | +| mdnsServer | | *[MdnsServer](#mdnsserver) | | false | +| cmServer | | *[CmServer](#cmserver) | | false | +| statsArchiver | | *[StatsArchiver](#statsarchiver) | | false | +| saslauthd | | *[Saslauthd](#saslauthd) | | false | +| pdnsServer | | *[PdnsServer](#pdnsserver) | | false | +| crdbCoordinator | | *[CrdbCoordinator](#crdbcoordinator) | | false | +| crdbWorker | | *[CrdbWorker](#crdbworker) | | false | +[Back to Table of Contents](#table-of-contents) + +### Saslauthd + + +| Field | Description | Scheme | Default Value | Required | +| ----- | ----------- | ------ | -------- | -------- | +| operatingMode | Whether to enable/disable the saslauthd service | [OperatingMode](#operatingmode) | | true | +[Back to Table of Contents](#table-of-contents) + ### ServicesRiggerConfigurationSpec Specification for service rigger @@ -162,6 +238,14 @@ Specification for service rigger | slaveHAGracePeriod | Time in seconds between when a node fails, and when slave high availability mechanism starts relocating shards. If set to 0, will not affect cluster configuration. | *uint32 | 1800 | true | [Back to Table of Contents](#table-of-contents) +### StatsArchiver + + +| Field | Description | Scheme | Default Value | Required | +| ----- | ----------- | ------ | -------- | -------- | +| operatingMode | Whether to enable/disable the stats archiver service | [OperatingMode](#operatingmode) | | true | +[Back to Table of Contents](#table-of-contents) + ### UpgradeSpec Specification for upgrades of Redis Enterprise @@ -207,6 +291,14 @@ State of the Redis Enterprise Cluster | "Deleting" | ClusterDeleting | [Back to Table of Contents](#table-of-contents) +### OperatingMode + +| Value | Description | +| ----- | ----------- | +| "enabled" | | +| "disabled" | | +[Back to Table of Contents](#table-of-contents) + ### SpecStatusName Whether the REC specification is valid (custom resource) diff --git a/redis_enterprise_database_api.md b/redis_enterprise_database_api.md index 609acb6..65e3025 100644 --- a/redis_enterprise_database_api.md +++ b/redis_enterprise_database_api.md @@ -35,7 +35,7 @@ This document describes the parameters for the Redis Enterprise Database custom | Field | Description | Scheme | Default Value | Required | | ----- | ----------- | ------ | -------- | -------- | -| absSecretName | The name of the K8s secret that holds ABS credentials. The secret must contain the keys \"AccountName\" and \"AccountKey\", and these must hold the corresponding credentials | string | | true | +| absSecretName | The name of the secret that holds ABS credentials. The secret must contain the keys \"AccountName\" and \"AccountKey\", and these must hold the corresponding credentials | string | | true | | container | Azure Blob Storage container name. | string | | true | | subdir | Optional. Azure Blob Storage subdir under container. | string | empty | false | [Back to Table of Contents](#table-of-contents) @@ -108,7 +108,7 @@ GoogleStorage | Field | Description | Scheme | Default Value | Required | | ----- | ----------- | ------ | -------- | -------- | -| gcsSecretName | The name of the K8s secret that holds the Google Cloud Storage credentials. The secret must contain the keys \"CLIENT_ID\", \"PRIVATE_KEY\", \"PRIVATE_KEY_ID\", \"CLIENT_EMAIL\" and these must hold the corresponding credentials. The keys should correspond to the values in the key JSON. | string | | true | +| gcsSecretName | The name of the secret that holds the Google Cloud Storage credentials. The secret must contain the keys \"CLIENT_ID\", \"PRIVATE_KEY\", \"PRIVATE_KEY_ID\", \"CLIENT_EMAIL\" and these must hold the corresponding credentials. The keys should correspond to the values in the key JSON. | string | | true | | bucketName | Google Storage bucket name. | string | | true | | subdir | Optional. Google Storage subdir under bucket. | string | empty | false | [Back to Table of Contents](#table-of-contents) @@ -168,7 +168,7 @@ RedisEnterpriseDatabaseSpec defines the desired state of RedisEnterpriseDatabase | shardCount | Number of database server-side shards | uint16 | 1 | false | | replication | In-memory database replication. When enabled, database will have replica shard for every master - leading to higher availability. | *bool | false | false | | persistence | Database on-disk persistence policy | *[DatabasePersistence](#databasepersistence) | disabled | false | -| databaseSecretName | The name of the K8s secret that holds the password to the database. | string | | false | +| databaseSecretName | The name of the secret that holds the password to the database. | string | | false | | evictionPolicy | Database eviction policy. see more https://docs.redislabs.com/latest/rs/administering/database-operations/eviction-policy/ | string | volatile-lru | false | | tlsMode | Require SSL authenticated and encrypted connections to the database. enabled - all incoming connections to the Database must use SSL. disabled - no incoming connection to the Database should use SSL. replica_ssl - databases that replicate from this one need to use SSL. | string | disabled | false | | clientAuthenticationCertificates | The Secrets containing TLS Client Certificate to use for Authentication | []string | | false | @@ -178,6 +178,8 @@ RedisEnterpriseDatabaseSpec defines the desired state of RedisEnterpriseDatabase | modulesList | List of modules associated with database | *[][DbModule](#dbmodule) | | false | | rolesPermissions | List of Redis Enteprise ACL and Role bindings to apply | [][RolePermission](#rolepermission) | | false | | defaultUser | Is connecting with a default user allowed? If disabled, the DatabaseSecret will not be created or updated | *bool | true | false | +| ossCluster | OSS Cluster mode option. Note that not all client libraries support OSS cluster mode. | *bool | false | false | +| proxyPolicy | The policy used for proxy binding to the endpoint. Supported proxy policies are: single/all-master-shards/all-nodes When left blank, the default value will be chosen according to the value of ossCluster - single if disabled, all-master-shards when enabled | string | | false | [Back to Table of Contents](#table-of-contents) ### RedisEnterpriseDatabaseStatus @@ -205,8 +207,8 @@ RedisEnterpriseDatabaseStatus defines the observed state of RedisEnterpriseDatab | Field | Description | Scheme | Default Value | Required | | ----- | ----------- | ------ | -------- | -------- | -| replicaSourceType | Determines what Kuberetes resource ReplicaSourceName refers to SECRET - Get URI from secret named in ReplicaSourceName. The secret will have a uri key that defines the complete, redis:// URI REDB - Determine URI from Kubernetes REDB resource named in ReplicaSourceName | [RepliceSourceType](#replicesourcetype) | | true | -| replicaSourceName | Kubernetes resource (SECRET/REDB) name of type ReplicaSourceType | string | | true | +| replicaSourceType | Determines what resource ReplicaSourceName refers to SECRET - Get URI from secret named in ReplicaSourceName. The secret will have a key named 'uri' that defines the complete, redis:// URI. The type of secret is determined by the secret mechanism used by the underlying REC object REDB - Determine URI from Kubernetes REDB resource named in ReplicaSourceName | [RepliceSourceType](#replicesourcetype) | | true | +| replicaSourceName | Resource (SECRET/REDB) name of type ReplicaSourceType | string | | true | | compression | GZIP Compression level (0-9) to use for replication | int | | false | | clientKeySecret | Secret that defines what client key to use. The secret needs 2 keys in its map, \"cert\" that is the PEM encoded certificate and \"key\" that is the PEM encoded private key | *string | | false | | serverCertSecret | Secret that defines the Server's certificate. The secret needs 1 key in its map, \"cert\" that is the PEM encoded certificate | *string | | false | @@ -242,7 +244,7 @@ Redis Enterprise Role and ACL Binding | Field | Description | Scheme | Default Value | Required | | ----- | ----------- | ------ | -------- | -------- | -| awsSecretName | The name of the K8s secret that holds the AWS credentials. The secret must contain the keys \"AWS_ACCESS_KEY_ID\" and \"AWS_SECRET_ACCESS_KEY\", and these must hold the corresponding credentials. | string | | true | +| awsSecretName | The name of the secret that holds the AWS credentials. The secret must contain the keys \"AWS_ACCESS_KEY_ID\" and \"AWS_SECRET_ACCESS_KEY\", and these must hold the corresponding credentials. | string | | true | | bucketName | Amazon S3 bucket name. | string | | true | | subdir | Optional. Amazon S3 subdir under bucket. | string | empty | false | [Back to Table of Contents](#table-of-contents) @@ -252,7 +254,7 @@ Redis Enterprise Role and ACL Binding | Field | Description | Scheme | Default Value | Required | | ----- | ----------- | ------ | -------- | -------- | -| sftpSecretName | The name of the K8s secret that holds SFTP credentials. The secret must contain the \"Key\" key, which is the SSH private key for connecting to the sftp server. | string | | true | +| sftpSecretName | The name of the secret that holds SFTP credentials. The secret must contain the \"Key\" key, which is the SSH private key for connecting to the sftp server. | string | | true | | sftp_url | SFTP url | string | | true | [Back to Table of Contents](#table-of-contents) @@ -261,7 +263,7 @@ Redis Enterprise Role and ACL Binding | Field | Description | Scheme | Default Value | Required | | ----- | ----------- | ------ | -------- | -------- | -| swiftSecretName | The name of the K8s secret that holds Swift credentials. The secret must contain the keys \"Key\" and \"User\", and these must hold the corresponding credentials: service access key and service user name (pattern for the latter does not allow special characters &,<,>,\") | string | | true | +| swiftSecretName | The name of the secret that holds Swift credentials. The secret must contain the keys \"Key\" and \"User\", and these must hold the corresponding credentials: service access key and service user name (pattern for the latter does not allow special characters &,<,>,\") | string | | true | | auth_url | Swift service authentication URL. | string | | true | | container | Swift object store container for storing the backup files. | string | | true | | prefix | Optional. Prefix (path) of backup files in the swift container. | string | empty | false | diff --git a/release_info.yaml b/release_info.yaml index be06515..66f3667 100644 --- a/release_info.yaml +++ b/release_info.yaml @@ -1,2 +1,2 @@ -operatorVersion: 6.0.12-5 -rsVersion: 6.0.12-57 +operatorVersion: 6.0.20-4 +rsVersion: 6.0.20-69 diff --git a/role.yaml b/role.yaml index 698941c..f082444 100644 --- a/role.yaml +++ b/role.yaml @@ -20,7 +20,7 @@ rules: verbs: ["get", "list", "watch"] - apiGroups: [""] resources: ["events"] - verbs: ["create"] + verbs: ["create", "patch"] - apiGroups: ["apps"] resources: ["deployments", "statefulsets", "replicasets"] verbs: ["*"] diff --git a/vault/README.md b/vault/README.md new file mode 100644 index 0000000..c8436f9 --- /dev/null +++ b/vault/README.md @@ -0,0 +1,240 @@ +# Integrating the Redis Enterprise Operator with Hashicorp Vault +## Overview +Hashicorp Vault can be used to store secrets as an alternative to K8s secrets. Hashicorp Vault can be configured as the source of secrets used by the Redis Enterprise K8s operator. For now, the following items are supported: +* Redis Enterprise Cluster Credentials +* REDB admission TLS cert +* REDB secrets: + * Replica of + * Backup credentials + * TLS keys + * default user secret + +This document explains how to use Hashicorp Vault as a source for secrets. +> Note: when using Openshift it might be recommended to use oc instead of kubectl +## Prerequisites +* Deploy a Hashicorp Vault instance and make sure there is network access to it from the Kubernetes cluster. The solution has been tested with Hashicorp Vault v1.6.2. The Hashicorp Vault instance must be using TLS. +* Configure the Hashicorp Vault Kubernetes authentication for the Kubernetes cluster the operator is being deployed. Refer to the Hashicorp Vault documentation for details. +* Deploy the Hashicorp Vault agent sidecar controller on the Kubernetes cluster (https://learn.hashicorp.com/tutorials/vault/kubernetes-sidecar) +* Note that Hashicorp offers a Vault Enterprise product. The Vault Enterprise product supports namespaces. Those namespaces should not be confused with Kubernetes namespaces. This document assumes that the Hashicorp Vault instance used is the Enterprise product and a Vault namespace is used. The namespace is referred to as the below. +* Redis Enterprise will use a kv-v2 secret engine. Make sure it is available on the Hashicorp Vault instance (or create one if needed) and take note of the path it is mounted on, since it will be used later. + +## Deployment +### General considerations +Hashicorp Vault and the Redis Enterprise Operator can be deployed in multiple scenarios that might affect the details of the process below. The document assumes the following: +* Hashicorp Vault enterprise is used, and Vault namespaces are used. If that is not the case, it is recommended to remove the namespace parameters, environment variables and annotations from the relevant directions. +* Multiple Redis Enterprise Clusters are configured within the same K8s cluster, configured to authenticate to Hashicorp Vault. +* To ensure privacy and avoid duplication, the K8S_NAMESPACE is appended to multiple names of Hashicorp Vault configurations. That might need to be further adjusted in cases multiple K8s clusters are used with the same K8s namespaces. +### Deploying the operator +1. Deploy the operator by applying the Redis Labs Kubernetes Operator Bundle as explained [here](../README.md) - steps 1,2 (steps 1-4 on OpenShift). Once operator is running, proceed to the steps below. Avoid creating the Redis Enterprise Cluster custom resource. +2. Configure a Hashicorp Vault policy. The policy will be used to grant the operator access to the secrets. + + Run the following command within the Hashicorp Vault interface (use kubectl exec when Vault is deployed on Kubernetes, replace with the namespace where the operator is deployed into): + ``` + vault policy write -namespace= redisenterprise- - </*" { + capabilities = ["create", "read", "update", "delete", "list"] + } + path "secret/metadata/redisenterprise-/*" { + capabilities = ["list"] + } + EOF + ``` +3. Create a role to bind the Redis Enterprise operator service account to the policy configured in the previous step: + ``` + vault write -namespace= auth//role/redis-enterprise-operator- \ + bound_service_account_names="redis-enterprise-operator" \ + bound_service_account_namespaces= \ + policies=redisenterprise- + ``` + > Note - replace AUTH_PATH with the path kubernetes auth is enabled in Hashicorp Vault. The default is "kubernetes" +4. Create a K8s secret containing the Certificate Authority Certificate (CACert) used to create the Hashicorp Vault instance server certificate. Name the secret vault-ca-cert. Save the ca cert to a file before running the following command: +``` +kubectl create secret generic vault-ca-cert \ + --namespace \ + --from-file=vault.ca= +``` +5. Modify the operator deployment to enable Hashicorp Vault agent sidecar container: + * Determine the Hashicorp Vault server Fully Qualified Domain Name (FQDN). If the Vault server is running with k8s, it would typically be .): + * The path the kv-2 secret engine being used is enabled on should be set as the value of the "VAULT_SECRET_ROOT" environment variable. + * The value of the VAULT_SECRET_PREFIX should be unique to the Redis Enterprise Cluster. Here we use "redisenterprise-". This value has to be consistent with Hashicorp Vault roles and policies. + * The value of the VAULT_SERVER_FQDN environment variable should be set with the Fully Qualified Domain Name of the Hashicorp Vault server. + * Save the following content to a file called operator-deployment-patch.yaml, replacing values as needed: +``` +spec: + template: + metadata: + annotations: + vault.hashicorp.com/agent-init-first: "true" + vault.hashicorp.com/agent-inject: "true" + vault.hashicorp.com/agent-inject-token: "true" + vault.hashicorp.com/ca-cert: "/vault/tls/vault.ca" + vault.hashicorp.com/tls-secret: "vault-ca-cert" + vault.hashicorp.com/role: "redis-enterprise-operator-" + vault.hashicorp.com/namespace: + vault.hashicorp.com/auth-path: auth/ + spec: + volumes: + - name: vault-ca-cert + secret: + defaultMode: 420 + secretName: vault-ca-cert + containers: + - name: redis-enterprise-operator + env: + - name: VAULT_SERVER_FQDN + value: + - name: VAULT_SERVICE_PORT_HTTPS + value: "8200" + - name: VAULT_SECRET_ROOT + value: "secret" + - name: VAULT_SECRET_PREFIX + value: "redisenterprise-" + - name: VAULT_NAMESPACE + value: + volumeMounts: + - mountPath: /vault/tls + name: vault-ca-cert + readOnly: true + ``` + > Note - the server certificate of the Hashicorp Vault instance must be signed by the Certificate Authority used within the secret. + > + * Run the following command to update the operator deployment: + ``` + kubectl patch deployment redis-enterprise-operator -n --patch "$(cat operator-deployment-patch.yaml)" + ``` + > Note - the change configures the sidecar injector to inject a token into the operator pod and configures the TLS settings required for secure communication with the Hashicorp Vault instance. +5. Verify that the sidecar container was created within the operator deployment. The operator pod should have 2 containers running. +### Creating the REC +1. Choose a random password. Unlike the default deployment, the operator is not creating a default password for the Redis Enterprise Cluster credentials and those need to be chosen. It is recommended to use a tool to generate a random password at least 8 characters long. +2. Save the password as a secret within the Hashicorp Vault instance, replace values as needed. Execute the following command within the Hashicorp Vault CLI interface: + ``` + vault kv put -namespace= /redisenterprise-/ username= password= + ``` + > Note - The username field in the REC spec will be ignored when using vault. The username from the vault secret will be used instead. + > Note - this example matches configuring the operator with environment variable values: VAULT_SECRET_ROOT=secret, VAULT_SECRET_PREFIX=redisenterprise- as mentioned above +3. Create a role in vault for the REC service account: + ``` + vault write -namespace= auth//role/redis-enterprise-rec- \ + bound_service_account_names= \ + bound_service_account_namespaces= \ + policies=redisenterprise- + + ``` +4. Apply the Redis Enterprise Cluster yaml. Example (make sure the clusterCredentialSecretName is consistent with Hashicorp Vault configuration above): + ``` + apiVersion: app.redislabs.com/v1 + kind: RedisEnterpriseCluster + metadata: + name: rec + spec: + # Add fields here + nodes: 3 + clusterCredentialSecretName: rec + clusterCredentialSecretType: vault + clusterCredentialSecretRole: redis-enterprise-rec- + vaultCASecret: vault-ca-cert + podAnnotations: + vault.hashicorp.com/namespace: + vault.hashicorp.com/auth-path: auth/ + + ``` + > Note - the "clusterCredentialSecretName" field as used to query the secret from Hashicorp Vault. See section below for explanation about secret name field values. +### Deploy REDB admission controller +It is not recommended to use the admission bundle here if you want to avoid creation of K8s secrets. +Instead, do a step by step installation. +1. Deploy the service - apply the following [yaml](../admission/service.yaml) +2. Generate a json file with key/cert pair to be used by admission: + ``` + kubectl exec -it $(kubectl get pod -l name=redis-enterprise-operator -o jsonpath='{.items[0].metadata.name}') -- /usr/local/bin/generate-tls -infer > output.json + ``` +3. Apply the secret to vault - execute the following within the Hashicorp Vault CLI interface (you will need to copy the file from the previous step): + ``` + vault kv put secret/redisenterprise/admission-tls @output.json + ``` +4. Apply the admission deployment [yaml](../admission/deployment.yaml) +5. Modify the admission deployment to enable Hashicorp Vault agent sidecar container: + * Save the following content to a file called admission-deployment-patch.yaml (set values as needed): + ``` + spec: + template: + metadata: + annotations: + vault.hashicorp.com/agent-init-first: "true" + vault.hashicorp.com/agent-inject: "true" + vault.hashicorp.com/agent-inject-token: "true" + vault.hashicorp.com/ca-cert: "/vault/tls/vault.ca" + vault.hashicorp.com/tls-secret: "vault-ca-cert" + vault.hashicorp.com/role: "redis-enterprise-operator-" + vault.hashicorp.com/namespace: + vault.hashicorp.com/auth-path: auth/ + spec: + serviceAccountName: redis-enterprise-operator + volumes: + - name: vault-ca-cert + secret: + defaultMode: 420 + secretName: vault-ca-cert + containers: + - name: admin + env: + - name: VAULT_SERVER_FQDN + value: + - name: VAULT_SERVICE_PORT_HTTPS + value: "8200" + - name: CREDENTIAL_TYPE + value: "vault" + - name: VAULT_SECRET_ROOT + value: "secret" + - name: VAULT_SECRET_PREFIX + value: "redisenterprise-" + - name: VAULT_NAMESPACE + value: + volumeMounts: + - mountPath: /vault/tls + name: vault-ca-cert + readOnly: true + ``` + * Patch the deployment + ``` + kubectl patch deployment admission-deploy -n --patch "$(cat admission-deployment-patch.yaml)" + ``` +6. Create the Kubernetes Validating Webhook + **NOTE**: One must replace REPLACE_WITH_NAMESPACE in the following command with the namespace the REC was installed into. + + ```shell script + # save cert + CERT=`cat output.json | jq -r ".cert"` + sed 's/NAMESPACE_OF_SERVICE_ACCOUNT/REPLACE_WITH_NAMESPACE/g' ../admission/webhook.yaml | kubectl create -f - + + # create patch file + cat > modified-webhook.yaml </redb-`:
+ e.g. ```vault kv put secret/redisenterprise-/redb-mydb password=somepassword``` +2. Create the REDB custom resource. + Follow the step 6 [here](../README.md). + The REC spec indicted you are running with Vault and no further configuration is required. +3. The other REDB secrets (2 to 4) should be created in this path `redisenterprise-/`. The secrets should comply with the + REDB [secrets schema](https://github.com/RedisLabs/redis-enterprise-operator/blob/master/deploy/redis_enterprise_database_api.md). +> Note - when using the Redis Enterprise Vault plugin it it recommended to set defaultUser: false and associate users through ACL bindings to the REDB + + diff --git a/vault/cluster_role_binding.yaml b/vault/cluster_role_binding.yaml new file mode 100644 index 0000000..dd6fc1b --- /dev/null +++ b/vault/cluster_role_binding.yaml @@ -0,0 +1,15 @@ +kind: ClusterRoleBinding +apiVersion: rbac.authorization.k8s.io/v1 +metadata: + name: redis-enterprise-vault-auth +subjects: +- kind: ServiceAccount + name: redis-enterprise-operator + namespace: NAMESPACE_OF_SERVICE_ACCOUNT +- kind: ServiceAccount + name: redis-enterprise-admission + namespace: NAMESPACE_OF_SERVICE_ACCOUNT +roleRef: + kind: ClusterRole + name: system:auth-delegator + apiGroup: rbac.authorization.k8s.io