From f1990e950b503412677945689aa0174d20c69326 Mon Sep 17 00:00:00 2001
From: "github-actions[bot]"
<41898282+github-actions[bot]@users.noreply.github.com>
Date: Mon, 11 Sep 2023 16:10:26 -0700
Subject: [PATCH] Document serviceDomain parameter in k8s config: PCP-935
(#1527) (#1537)
* Added serviceDomain to first tab only
* Add serviceDomain to all version tabs
* Removed breaks as needed, indented yaml
* Fixed typo
* Removed unneeded 'the' before link.
(cherry picked from commit 62e20d4a0fbfb2038650eeeb478364d24e6c7cc8)
Co-authored-by: Rita Watson <117382432+ritawatson@users.noreply.github.com>
---
.../integrations/kubernetes-generic.md | 253 ++++-----
docs/docs-content/integrations/kubernetes.md | 506 +++++++++---------
2 files changed, 372 insertions(+), 387 deletions(-)
diff --git a/docs/docs-content/integrations/kubernetes-generic.md b/docs/docs-content/integrations/kubernetes-generic.md
index dd99ff399c..6adb14e32b 100644
--- a/docs/docs-content/integrations/kubernetes-generic.md
+++ b/docs/docs-content/integrations/kubernetes-generic.md
@@ -53,6 +53,7 @@ Review the [Maintenance Policy](/integrations/maintenance-policy) to learn about
| `pack.palette.config.oidcidentityProvider`| OIDC identity provider configuration. |
| ``pack.podCIDR`` | The CIDR range for Pods in cluster. This should match the networking layer property. Default: `192.168.0.0/16`|
| ``pack.serviceClusterIpRange`` | The CIDR range for services in the cluster. This should not overlap with any IP ranges assigned to nodes or pods. Default: `10.96.0.0/12`|
+| `pack.serviceDomain` | The cluster DNS service domain. Default: `cluster.local`. To change the default, you must add this parameter to the Kubernetes YAML file at cluster creation and specify the cluster DNS service domain to use. This value cannot be changed after cluster creation is complete. Refer to the [Change Cluster DNS Service Domain](/integrations/kubernetes-generic?platform=AKS&versions=k8s_v1.27#change-cluster-dns-service-domain) section. |
| ``kubeadmconfig.apiServer.extraArgs`` | A list of additional apiServer flags you can set.|
| ``kubeadmconfig.apiServer.extraVolumes`` | A list of additional volumes to mount on apiServer.|
| ``kubeadmconfig.controllerManager.extraArgs`` | A list of additional ControllerManager flags to set.|
@@ -61,46 +62,53 @@ Review the [Maintenance Policy](/integrations/maintenance-policy) to learn about
| ``kubeadmconfig.files`` | A list of additional files to copy to the nodes.|
| ``kubeadmconfig.preKubeadmCommands`` | A list of additional commands to invoke **before** running kubeadm commands.|
| ``kubeadmconfig.postKubeadmCommands`` | A list of additional commands to invoke **after** running kubeadm commands.|
-| ``pack.serviceDomain`` | The DNS name for the service domain in the cluster. Default: ``cluster.local``.|
## Usage
The Kubeadm configuration file is where you can do the following:
-
-
- Change the default ``podCIDR`` and ``serviceClusterIpRange`` values. CIDR IPs specified in the configuration file take precedence over other defined CIDR IPs in your environment.
- As you build your cluster, check that the ``podCIDR`` value does not overlap with any hosts or with the service network and the ``serviceClusterIpRange`` value does not overlap with any IP ranges assigned to nodes or pods. For more information, refer to the [Clusters](/clusters) guide and [Cluster Deployment Errors](https://docs.spectrocloud.com/troubleshooting/cluster-deployment).
+ As you build your cluster, check that the ``podCIDR`` value does not overlap with any hosts or with the service network and the ``serviceClusterIpRange`` value does not overlap with any IP ranges assigned to nodes or pods. For more information, refer to the [Clusters](/clusters) guide and [Cluster Deployment Errors](https://docs.spectrocloud.com/troubleshooting/cluster-deployment).
+- Change the default cluster DNS service domain from `cluster.local` to a DNS domain that you specify. You can only change the DNS domain during cluster creation. For more information, refer to [Change Cluster DNS Service Domain](/integrations/kubernetes-generic?platform=AKS&versions=k8s_v1.27#change-cluster-dns-service-domain).
- Add a certificate for the Spectro Proxy pack if you want to use a reverse proxy with a Kubernetes cluster. For more information, refer to the [Spectro Proxy](/integrations/frp) guide.
+### Change Cluster DNS Service Domain
-#### Configuration Changes
+The `pack.serviceDomain` parameter with default value `cluster.local` is not visible in the Kubernetes YAML file, and its value can only be changed at cluster creation. To change the value, you must add `serviceDomain: "cluster.local"` to the Kubernetes YAML file when you create a cluster, and specify the service domain you want to use.
-The Kubeadm config is updated with hardening improvements that do the following:
+```yaml hideClipboard
+pack:
+ k8sHardening: True
+ podCIDR: "172.16.0.0/16"
+ serviceClusterIPRange: "10.96.0.0/12"
+ serviceDomain: ""
+```
-
+:::caution
-- Meet CIS standards for operating systems (OS).
+You can only specify the service domain at cluster creation. After cluster creation completes, you cannot update the value. Attempting to update it results in the error `serviceDomain update is forbidden for existing cluster`.
+:::
-- Enable a Kubernetes audit policy in the pack. The audit policy is hidden, and you cannot customize the default audit policy. If you want to apply your custom audit policy, refer to the [Enable Audit Logging](/audit-logs/kube-api-audit-logging) guide to learn how to create your custom audit policy by adjusting API server flags.
+For more information about networking configuration with DNS domains, refer to the Kubernetes [Networking](https://kubernetes.io/docs/reference/config-api/kubeadm-config.v1beta3/#kubeadm-k8s-io-v1beta3-Networking) API documentation.
+### Configuration Changes
-- Replace a deprecated PodSecurityPolicy (PSP) with one that offers three built-in policy profiles for broad security coverage:
+The Kubeadm config is updated with hardening improvements that do the following:
-
+- Meet CIS standards for operating systems (OS).
- - **Privileged**: An unrestricted policy that provides wide permission levels and allows for known privilege escalations.
+- Enable a Kubernetes audit policy in the pack. The audit policy is hidden, and you cannot customize the default audit policy. If you want to apply your custom audit policy, refer to the [Enable Audit Logging](/audit-logs/kube-api-audit-logging) guide to learn how to create your custom audit policy by adjusting API server flags.
-
+- Replace a deprecated PodSecurityPolicy (PSP) with one that offers three built-in policy profiles for broad security coverage:
- - **Baseline**: A policy that offers minimal restrictions and prevents known privilege escalations. As shown in the example below, you can override the default cluster-wide policy to set baseline enforcement by enabling the `PodSecurity` Admission plugin in the `enable-admission-plugins` section of the YAML file. You can then add a custom Admission configuration and set the `admission-control-config-file` flag to the custom Admission.
+ - **Privileged**: An unrestricted policy that provides wide permission levels and allows for known privilege escalations.
-
+ - **Baseline**: A policy that offers minimal restrictions and prevents known privilege escalations. As shown in the example below, you can override the default cluster-wide policy to set baseline enforcement by enabling the `PodSecurity` Admission plugin in the `enable-admission-plugins` section of the YAML file. You can then add a custom Admission configuration and set the `admission-control-config-file` flag to the custom Admission.
```yaml
kubeadmconfig:
@@ -120,21 +128,15 @@ The Kubeadm config is updated with hardening improvements that do the following:
- **Restricted**: A heavily restricted policy that follows Pod hardening best practices. This policy is set to warn and audit and identifies Pods that require privileged access.
-
-
You can enforce these policies at the cluster level or the Namespace level. For workloads that require privileged access, you can relax `PodSecurity` enforcement by adding these labels in the Namespace:
-
-
```yaml
pod-security.kubernetes.io/enforce: privileged
pod-security.kubernetes.io/enforce-version: v1.26
```
-
-
-#### Kubeadm Configuration File
+### Kubeadm Configuration File
The default pack YAML contains minimal configurations offered by the managed provider.
@@ -145,14 +147,11 @@ You can configure an OpenID Connect (OIDC) identity provider to authenticate use
OIDC requires a *RoleBinding* for the users or groups you want to provide cluster access. You must create a RoleBinding to a Kubernetes role that is available in the cluster. The Kubernetes role can be a custom role you created or a [default Kubernetes role](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles), such as the `cluster-admin` role. To learn how to create a RoleBinding through Palette, refer to [Create Role Bindings](/clusters/cluster-management/cluster-rbac/#createrolebindings).
-
#### Configure Custom OIDC
The custom method to configure OIDC and apply RBAC for an OIDC provider can be used for all cloud services except Amazon Elastic Kubernetes Service (EKS) and [Azure-AKS](/clusters/public-cloud/azure/aks/#configureanazureactivedirectory).
-
-
@@ -160,13 +159,9 @@ The custom method to configure OIDC and apply RBAC for an OIDC provider can be u
Follow these steps to configure a third-party OIDC IDP. You can apply these steps to all the public cloud providers except Azure AKS and Amazon EKS clusters. Azure AKS and Amazon EKS require different configurations. AKS requires you to use Azure Active Directory (AAD) to enable OIDC integration. Refer to [Azure-AKS](/clusters/public-cloud/azure/aks/#configureanazureactivedirectory) to learn more. Click the **Amazon EKS** tab for steps to configure OIDC for EKS clusters.
-
-
1. Add the following parameters to your Kubernetes YAML file when creating a cluster profile. Replace the `identityProvider` value with your OIDC provider name.
-
-
```yaml
pack:
palette:
@@ -177,8 +172,6 @@ Follow these steps to configure a third-party OIDC IDP. You can apply these step
2. Add the following `kubeadmconfig` parameters. Replace the values with your OIDC provider values.
-
-
```yaml
kubeadmconfig:
apiServer:
@@ -191,8 +184,6 @@ Follow these steps to configure a third-party OIDC IDP. You can apply these step
3. Under the `clientConfig` parameter section of Kubernetes YAML file, uncomment the `oidc-` configuration lines.
-
-
```yaml
kubeadmconfig:
clientConfig:
@@ -272,6 +263,7 @@ clientConfig:
| `pack.palette.config.oidcidentityProvider`| OIDC identity provider configuration. |
| ``pack.podCIDR`` | The CIDR range for Pods in cluster. This should match the networking layer property. Default: `192.168.0.0/16`|
| ``pack.serviceClusterIpRange`` | The CIDR range for services in the cluster. This should not overlap with any IP ranges assigned to nodes or pods. Default: `10.96.0.0/12`|
+| `pack.serviceDomain` | The cluster DNS service domain. Default: `cluster.local`. To change the default, you must add this parameter to the Kubernetes YAML file at cluster creation and specify the cluster DNS service domain to use. This value cannot be changed after cluster creation is complete. Refer to the [Change Cluster DNS Service Domain](/integrations/kubernetes-generic?platform=AKS&versions=k8s_v1.26#change-cluster-dns-service-domain-1) section. |
| ``kubeadmconfig.apiServer.extraArgs`` | A list of additional apiServer flags you can set.|
| ``kubeadmconfig.apiServer.extraVolumes`` | A list of additional volumes to mount on apiServer.|
| ``kubeadmconfig.controllerManager.extraArgs`` | A list of additional ControllerManager flags to set.|
@@ -280,46 +272,54 @@ clientConfig:
| ``kubeadmconfig.files`` | A list of additional files to copy to the nodes.|
| ``kubeadmconfig.preKubeadmCommands`` | A list of additional commands to invoke **before** running kubeadm commands.|
| ``kubeadmconfig.postKubeadmCommands`` | A list of additional commands to invoke **after** running kubeadm commands.|
-| ``pack.serviceDomain`` | The DNS name for the service domain in the cluster. Default: ``cluster.local``.|
## Usage
The Kubeadm configuration file is where you can do the following:
-
-
- Change the default ``podCIDR`` and ``serviceClusterIpRange`` values. CIDR IPs specified in the configuration file take precedence over other defined CIDR IPs in your environment.
As you build your cluster, check that the ``podCIDR`` value does not overlap with any hosts or with the service network and the ``serviceClusterIpRange`` value does not overlap with any IP ranges assigned to nodes or pods. For more information, refer to the [Clusters](/clusters) guide and [Cluster Deployment Errors](https://docs.spectrocloud.com/troubleshooting/cluster-deployment).
+- Change the default cluster DNS service domain from `cluster.local` to a DNS domain that you specify. You can only change the DNS domain during cluster creation. For more information, refer to [Change Cluster DNS Service Domain](/integrations/kubernetes-generic?platform=AKS&versions=k8s_v1.26#change-cluster-dns-service-domain-1).
- Add a certificate for the Spectro Proxy pack if you want to use a reverse proxy with a Kubernetes cluster. For more information, refer to the [Spectro Proxy](/integrations/frp) guide.
+### Change Cluster DNS Service Domain
-#### Configuration Changes
+The `pack.serviceDomain` parameter with default value `cluster.local` is not visible in the Kubernetes YAML file, and its value can only be changed at cluster creation. To change the value, you must add `serviceDomain: "cluster.local"` to the Kubernetes YAML file when you create a cluster, and specify the service domain you want to use.
-The Kubeadm config is updated with hardening improvements that do the following:
+```yaml
+pack:
+ k8sHardening: True
+ podCIDR: "172.16.0.0/16"
+ serviceClusterIPRange: "10.96.0.0/12"
+ serviceDomain: ""
+```
-
+:::caution
-- Meet CIS standards for operating systems (OS).
+You can only specify the service domain at cluster creation. After cluster creation completes, you cannot update the value. Attempting to update it results in the error `serviceDomain update is forbidden for existing cluster`.
+:::
-- Enable a Kubernetes audit policy in the pack. The audit policy is hidden, and you cannot customize the default audit policy. If you want to apply your custom audit policy, refer to the [Enable Audit Logging](/audit-logs/kube-api-audit-logging) guide to learn how to create your custom audit policy by adjusting API server flags.
+For more information about networking configuration with DNS domains, refer to the Kubernetes [Networking](https://kubernetes.io/docs/reference/config-api/kubeadm-config.v1beta3/#kubeadm-k8s-io-v1beta3-Networking) API documentation.
-- Replace a deprecated PodSecurityPolicy (PSP) with one that offers three built-in policy profiles for broad security coverage:
+### Configuration Changes
-
+The Kubeadm config is updated with hardening improvements that do the following:
- - **Privileged**: An unrestricted policy that provides wide permission levels and allows for known privilege escalations.
+- Meet CIS standards for operating systems (OS).
-
+- Enable a Kubernetes audit policy in the pack. The audit policy is hidden, and you cannot customize the default audit policy. If you want to apply your custom audit policy, refer to the [Enable Audit Logging](/audit-logs/kube-api-audit-logging) guide to learn how to create your custom audit policy by adjusting API server flags.
- - **Baseline**: A policy that offers minimal restrictions and prevents known privilege escalations. As shown in the example below, you can override the default cluster-wide policy to set baseline enforcement by enabling the `PodSecurity` Admission plugin in the `enable-admission-plugins` section of the YAML file. You can then add a custom Admission configuration and set the `admission-control-config-file` flag to the custom Admission.
+- Replace a deprecated PodSecurityPolicy (PSP) with one that offers three built-in policy profiles for broad security coverage:
-
+ - **Privileged**: An unrestricted policy that provides wide permission levels and allows for known privilege escalations.
+
+ - **Baseline**: A policy that offers minimal restrictions and prevents known privilege escalations. As shown in the example below, you can override the default cluster-wide policy to set baseline enforcement by enabling the `PodSecurity` Admission plugin in the `enable-admission-plugins` section of the YAML file. You can then add a custom Admission configuration and set the `admission-control-config-file` flag to the custom Admission.
```yaml
kubeadmconfig:
@@ -339,21 +339,14 @@ The Kubeadm config is updated with hardening improvements that do the following:
- **Restricted**: A heavily restricted policy that follows Pod hardening best practices. This policy is set to warn and audit and identifies Pods that require privileged access.
-
-
You can enforce these policies at the cluster level or the Namespace level. For workloads that require privileged access, you can relax `PodSecurity` enforcement by adding these labels in the Namespace:
-
-
```yaml
pod-security.kubernetes.io/enforce: privileged
pod-security.kubernetes.io/enforce-version: v1.26
```
-
-
-
-#### Kubeadm Configuration File
+### Kubeadm Configuration File
The default pack YAML contains minimal configurations offered by the managed provider.
@@ -364,14 +357,10 @@ You can configure an OpenID Connect (OIDC) identity provider to authenticate use
OIDC requires a *RoleBinding* for the users or groups you want to provide cluster access. You must create a RoleBinding to a Kubernetes role that is available in the cluster. The Kubernetes role can be a custom role you created or a [default Kubernetes role](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles), such as the `cluster-admin` role. To learn how to create a RoleBinding through Palette, refer to [Create Role Bindings](/clusters/cluster-management/cluster-rbac/#createrolebindings).
-
-
#### Configure Custom OIDC
The custom method to configure OIDC and apply RBAC for an OIDC provider can be used for all cloud services except Amazon Elastic Kubernetes Service (EKS) and [Azure-AKS](/clusters/public-cloud/azure/aks/#configureanazureactivedirectory).
-
-
@@ -379,9 +368,6 @@ The custom method to configure OIDC and apply RBAC for an OIDC provider can be u
Follow these steps to configure a third-party OIDC IDP. You can apply these steps to all the public cloud providers except Azure AKS and Amazon EKS clusters. Azure AKS and Amazon EKS require different configurations. AKS requires you to use Azure Active Directory (AAD) to enable OIDC integration. Refer to [Azure-AKS](/clusters/public-cloud/azure/aks/#configureanazureactivedirectory) to learn more. Click the **Amazon EKS** tab for steps to configure OIDC for EKS clusters.
-
-
-
1. Add the following parameters to your Kubernetes YAML file when creating a cluster profile. Replace the `identityProvider` value with your OIDC provider name.
@@ -396,8 +382,6 @@ Follow these steps to configure a third-party OIDC IDP. You can apply these step
2. Add the following `kubeadmconfig` parameters. Replace the values with your OIDC provider values.
-
-
```yaml
kubeadmconfig:
apiServer:
@@ -410,8 +394,6 @@ Follow these steps to configure a third-party OIDC IDP. You can apply these step
3. Under the `clientConfig` parameter section of Kubernetes YAML file, uncomment the `oidc-` configuration lines.
-
-
```yaml
kubeadmconfig:
clientConfig:
@@ -429,8 +411,6 @@ Follow these steps to configure a third-party OIDC IDP. You can apply these step
Follow these steps to configure OIDC for managed EKS clusters.
-
-
1. In the Kubernetes pack, uncomment the lines in the `oidcIdentityProvider` parameter section of the Kubernetes pack, and enter your third-party provider details.
```yaml
@@ -471,7 +451,6 @@ clientConfig:
- A minimum of 4 CPU and 4GB Memory.
-
- Operating System (OS) dependencies as listed in the table.
| OS Distribution | OS Version | Supports Kubernetes 1.25.x |
@@ -489,6 +468,7 @@ clientConfig:
| `pack.palette.config.oidcidentityProvider`| OIDC identity provider configuration. |
| ``pack.podCIDR`` | The CIDR range for Pods in cluster. This should match the networking layer property. Default: `192.168.0.0/16`|
| ``pack.serviceClusterIpRange`` | The CIDR range for services in the cluster. This should not overlap with any IP ranges assigned to nodes or pods. Default: `10.96.0.0/12`|
+| `pack.serviceDomain` | The cluster DNS service domain. Default: `cluster.local`. To change the default, you must add this parameter to the Kubernetes YAML file at cluster creation and specify the cluster DNS service domain to use. This value cannot be changed after cluster creation is complete. Refer to the [Change Cluster DNS Service Domain](/integrations/kubernetes-generic?versions=k8s_v1.25#change-cluster-dns-service-domain-2) section. |
| ``kubeadmconfig.apiServer.extraArgs`` | A list of additional apiServer flags you can set.|
| ``kubeadmconfig.apiServer.extraVolumes`` | A list of additional volumes to mount on apiServer.|
| ``kubeadmconfig.controllerManager.extraArgs`` | A list of additional ControllerManager flags to set.|
@@ -497,28 +477,42 @@ clientConfig:
| ``kubeadmconfig.files`` | A list of additional files to copy to the nodes.|
| ``kubeadmconfig.preKubeadmCommands`` | A list of additional commands to invoke **before** running kubeadm commands.|
| ``kubeadmconfig.postKubeadmCommands`` | A list of additional commands to invoke **after** running kubeadm commands.|
-| ``pack.serviceDomain`` | The DNS name for the service domain in the cluster. Default: ``cluster.local``.|
## Usage
The Kubeadm configuration file is where you can do the following:
-
-
- Change the default ``podCIDR`` and ``serviceClusterIpRange`` values. CIDR IPs specified in the configuration file take precedence over other defined CIDR IPs in your environment.
As you build your cluster, check that the ``podCIDR`` value does not overlap with any hosts or with the service network and the ``serviceClusterIpRange`` value does not overlap with any IP ranges assigned to nodes or pods. For more information, refer to the [Clusters](/clusters) guide and [Cluster Deployment Errors](https://docs.spectrocloud.com/troubleshooting/cluster-deployment).
+- Change the default cluster DNS service domain from `cluster.local` to a DNS domain that you specify. You can only change the DNS domain during cluster creation. For more information, refer to [Change Cluster DNS Service Domain](/integrations/kubernetes-generic?versions=k8s_v1.25#change-cluster-dns-service-domain-2).
- Add a certificate for the Spectro Proxy pack if you want to use a reverse proxy with a Kubernetes cluster. For more information, refer to the [Spectro Proxy](/integrations/frp) guide.
+### Change Cluster DNS Service Domain
-#### Configuration Changes
+The `pack.serviceDomain` parameter with default value `cluster.local` is not visible in the Kubernetes YAML file, and its value can only be changed at cluster creation. To change the value, you must add `serviceDomain: "cluster.local"` to the Kubernetes YAML file when you create a cluster, and specify the service domain you want to use.
+```yaml
+pack:
+ k8sHardening: True
+ podCIDR: "172.16.0.0/16"
+ serviceClusterIPRange: "10.96.0.0/12"
+ serviceDomain: ""
+```
-The Kubeadm config is updated with hardening improvements that do the following:
+:::caution
-
+You can only specify the service domain at cluster creation. After cluster creation completes, you cannot update the value. Attempting to update it results in the error `serviceDomain update is forbidden for existing cluster`.
+
+:::
+
+For more information about networking configuration with DNS domains, refer to the Kubernetes [Networking](https://kubernetes.io/docs/reference/config-api/kubeadm-config.v1beta3/#kubeadm-k8s-io-v1beta3-Networking) API documentation.
+
+### Configuration Changes
+
+The Kubeadm config is updated with hardening improvements that do the following:
- Meet CIS standards for operating systems (OS).
@@ -528,15 +522,10 @@ The Kubeadm config is updated with hardening improvements that do the following:
- Replace a deprecated PodSecurityPolicy (PSP) with one that offers three built-in policy profiles for broad security coverage:
-
-
- **Privileged**: An unrestricted policy that provides wide permission levels and allows for known privilege escalations.
-
-
- **Baseline**: A policy that offers minimal restrictions and prevents known privilege escalations. As shown in the example below, you can override the default cluster-wide policy to set baseline enforcement by enabling the `PodSecurity` Admission plugin in the `enable-admission-plugins` section of the YAML file. You can then add a custom Admission configuration and set the `admission-control-config-file` flag to the custom Admission.
-
```yaml
kubeadmconfig:
@@ -556,22 +545,14 @@ The Kubeadm config is updated with hardening improvements that do the following:
- **Restricted**: A heavily restricted policy that follows Pod hardening best practices. This policy is set to warn and audit and identifies Pods that require privileged access.
-
-
You can enforce these policies at the cluster level or the Namespace level. For workloads that require privileged access, you can relax `PodSecurity` enforcement by adding these labels in the Namespace:
-
-
- ```yaml
- pod-security.kubernetes.io/enforce: privileged
- pod-security.kubernetes.io/enforce-version: v1.25
- ```
-
-
-
-
+ ```yaml
+ pod-security.kubernetes.io/enforce: privileged
+ pod-security.kubernetes.io/enforce-version: v1.25
+ ```
-#### Kubeadm Configuration File
+### Kubeadm Configuration File
The default pack YAML contains minimal configurations offered by the managed provider.
@@ -582,14 +563,11 @@ You can configure an OpenID Connect (OIDC) identity provider to authenticate use
OIDC requires a *RoleBinding* for the users or groups you want to provide cluster access. You must create a RoleBinding to a Kubernetes role that is available in the cluster. The Kubernetes role can be a custom role you created or a [default Kubernetes role](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles), such as the `cluster-admin` role. To learn how to create a RoleBinding through Palette, refer to [Create Role Bindings](/clusters/cluster-management/cluster-rbac/#createrolebindings).
-
#### Configure Custom OIDC
The custom method to configure OIDC and apply RBAC for an OIDC provider can be used for all cloud services except Amazon Elastic Kubernetes Service (EKS) and [Azure-AKS](/clusters/public-cloud/azure/aks/#configureanazureactivedirectory).
-
-
@@ -597,12 +575,8 @@ The custom method to configure OIDC and apply RBAC for an OIDC provider can be u
Follow these steps to configure a third-party OIDC IDP. You can apply these steps to all the public cloud providers except Azure AKS and Amazon EKS clusters. Azure AKS and Amazon EKS require different configurations. AKS requires you to use Azure Active Directory (AAD) to enable OIDC integration. Refer to [Azure-AKS](/clusters/public-cloud/azure/aks/#configureanazureactivedirectory) to learn more. Click the **Amazon EKS** tab for steps to configure OIDC for EKS clusters.
-
-
1. Add the following parameters to your Kubernetes YAML file when creating a cluster profile. Replace the `identityProvider` value with your OIDC provider name.
-
-
```yaml
pack:
palette:
@@ -614,8 +588,6 @@ Follow these steps to configure a third-party OIDC IDP. You can apply these step
2. Add the following `kubeadmconfig` parameters. Replace the values with your OIDC provider values.
-
-
```yaml
kubeadmconfig:
apiServer:
@@ -625,19 +597,17 @@ Follow these steps to configure a third-party OIDC IDP. You can apply these step
oidc-groups-claim: "groups"
oidc-username-claim: "email"
```
-
3. Under the `clientConfig` parameter section of Kubernetes YAML file, uncomment the `oidc-` configuration lines.
-
-```yaml
-kubeadmconfig:
- clientConfig:
- oidc-issuer-url: ""
- oidc-client-id: ""
- oidc-client-secret: ""
- oidc-extra-scope: profile,email,openid
-```
+ ```yaml
+ kubeadmconfig:
+ clientConfig:
+ oidc-issuer-url: ""
+ oidc-client-id: ""
+ oidc-client-secret: ""
+ oidc-extra-scope: profile,email,openid
+ ```
@@ -710,6 +680,7 @@ Follow these steps to configure OIDC for managed EKS clusters.
| `pack.palette.config.oidcidentityProvider`| OIDC identity provider configuration. |
| `pack.podCIDR` | The CIDR range for Pods in cluster. This should match the networking layer property. Default: `192.168.0.0/16`|
| `pack.serviceClusterIpRange` | The CIDR range for services in the cluster. This should not overlap with any IP ranges assigned to nodes or pods. Default: `10.96.0.0/12`|
+| `pack.serviceDomain` | The cluster DNS service domain. Default: `cluster.local`. To change the default, you must add this parameter to the Kubernetes YAML file at cluster creation and specify the cluster DNS service domain to use. This value cannot be changed after cluster creation is complete. Refer to the [Change Cluster DNS Service Domain](/integrations/kubernetes-generic?versions=k8s_v1.24#change-cluster-dns-service-domain-3) section. |
| `kubeadmconfig.apiServer.extraArgs` | A list of additional apiServer flags you can set.|
| `kubeadmconfig.apiServer.extraVolumes` | A list of additional volumes to mount on apiServer.|
| `kubeadmconfig.controllerManager.extraArgs` | A list of additional ControllerManager flags to set.|
@@ -718,29 +689,45 @@ Follow these steps to configure OIDC for managed EKS clusters.
| `kubeadmconfig.files` | A list of additional files to copy to the nodes. |
| `kubeadmconfig.preKubeadmCommands` | A list of additional commands to invoke **before** running kubeadm commands.|
| `kubeadmconfig.postKubeadmCommands` | A list of additional commands to invoke **after** running kubeadm commands.|
-| `pack.serviceDomain` | The DNS name for the service domain in the cluster. Default: ``cluster.local``.|
-
## Usage
The Kubeadm configuration file is where you can do the following:
-
-
- Change the default ``podCIDR`` and ``serviceClusterIpRange`` values. CIDR IPs specified in the configuration file take precedence over other defined CIDR IPs in your environment.
As you build your cluster, check that the ``podCIDR`` value does not overlap with any hosts or with the service network and the ``serviceClusterIpRange`` value does not overlap with any IP ranges assigned to nodes or pods. For more information, refer to the [Clusters](/clusters) guide and [Cluster Deployment Errors](https://docs.spectrocloud.com/troubleshooting/cluster-deployment).
+- Change the default cluster DNS service domain from `cluster.local` to a DNS domain that you specify. You can only change the DNS domain during cluster creation. For more information, refer to [Change Cluster DNS Service Domain](/integrations/kubernetes-generic?versions=k8s_v1.24#change-cluster-dns-service-domain-3).
- Add a certificate for the Spectro Proxy pack if you want to use a reverse proxy with a Kubernetes cluster. For more information, refer to the [Spectro Proxy](/integrations/frp) guide.
+### Change Cluster DNS Service Domain
-#### Configuration Changes
+The `pack.serviceDomain` parameter with default value `cluster.local` is not visible in the Kubernetes YAML file, and its value can only be changed at cluster creation. To change the value, you must add `serviceDomain: "cluster.local"` to the Kubernetes YAML file when you create a cluster, and specify the service domain you want to use.
-The Kubeadm config is updated with hardening improvements that do the following:
+```yaml
+pack:
+ k8sHardening: True
+ podCIDR: "172.16.0.0/16"
+ serviceClusterIPRange: "10.96.0.0/12"
+ serviceDomain: ""
+```
+
+:::caution
+
+You can only specify the service domain at cluster creation. After cluster creation completes, you cannot update the value. Attempting to update it results in the error `serviceDomain update is forbidden for existing cluster`.
+
+:::
+
+For more information about networking configuration with DNS domains, refer to the Kubernetes [Networking](https://kubernetes.io/docs/reference/config-api/kubeadm-config.v1beta3/#kubeadm-k8s-io-v1beta3-Networking) API documentation.
+### Configuration Changes
+
+The Kubeadm config is updated with hardening improvements that do the following:
+
- Meet CIS standards for operating systems (OS).
@@ -749,16 +736,10 @@ The Kubeadm config is updated with hardening improvements that do the following:
- Replace a deprecated PodSecurityPolicy (PSP) with one that offers three built-in policy profiles for broad security coverage:
-
-
- **Privileged**: An unrestricted policy that provides wide permission levels and allows for known privilege escalations.
-
-
- **Baseline**: A policy that offers minimal restrictions and prevents known privilege escalations. As shown in the example below, you can override the default cluster-wide policy to set baseline enforcement by enabling the `PodSecurity` Admission plugin in the `enable-admission-plugins` section of the YAML file. You can then add a custom Admission configuration and set the `admission-control-config-file` flag to the custom Admission.
-
-
```yaml
kubeadmconfig:
apiServer:
@@ -777,22 +758,15 @@ The Kubeadm config is updated with hardening improvements that do the following:
- **Restricted**: A heavily restricted policy that follows Pod hardening best practices. This policy is set to warn and audit and identifies Pods that require privileged access.
-
-
You can enforce these policies at the cluster level or the Namespace level. For workloads that require privileged access, you can relax `PodSecurity` enforcement by adding these labels in the Namespace:
-
-
```yaml
pod-security.kubernetes.io/enforce: privileged
pod-security.kubernetes.io/enforce-version: v1.24
```
-
-
-
-#### Kubeadm Configuration File
+### Kubeadm Configuration File
The default pack YAML contains minimal configurations offered by the managed provider.
@@ -803,14 +777,11 @@ You can configure an OpenID Connect (OIDC) identity provider to authenticate use
OIDC requires a *RoleBinding* for the users or groups you want to provide cluster access. You must create a RoleBinding to a Kubernetes role that is available in the cluster. The Kubernetes role can be a custom role you created or a [default Kubernetes role](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles), such as the `cluster-admin` role. To learn how to create a RoleBinding through Palette, refer to [Create Role Bindings](/clusters/cluster-management/cluster-rbac/#createrolebindings).
-
#### Configure Custom OIDC
The custom method to configure OIDC and apply RBAC for an OIDC provider can be used for all cloud services except Amazon Elastic Kubernetes Service (EKS) and [Azure-AKS](/clusters/public-cloud/azure/aks/#configureanazureactivedirectory).
-
-
@@ -818,13 +789,9 @@ The custom method to configure OIDC and apply RBAC for an OIDC provider can be u
Follow these steps to configure a third-party OIDC IDP. You can apply these steps to all the public cloud providers except Azure AKS and Amazon EKS clusters. Azure AKS and Amazon EKS require different configurations. AKS requires you to use Azure Active Directory (AAD) to enable OIDC integration. Refer to [Azure-AKS](/clusters/public-cloud/azure/aks/#configureanazureactivedirectory) to learn more. Click the **Amazon EKS** tab for steps to configure OIDC for EKS clusters.
-
-
1. Add the following parameters to your Kubernetes YAML file when creating a cluster profile. Replace the `identityProvider` value with your OIDC provider name.
-
-
```yaml
pack:
palette:
@@ -836,8 +803,6 @@ Follow these steps to configure a third-party OIDC IDP. You can apply these step
2. Add the following `kubeadmconfig` parameters. Replace the values with your OIDC provider values.
-
-
```yaml
kubeadmconfig:
apiServer:
@@ -850,8 +815,6 @@ Follow these steps to configure a third-party OIDC IDP. You can apply these step
3. Under the `clientConfig` parameter section of Kubernetes YAML file, uncomment the `oidc-` configuration lines.
-
-
```yaml
kubeadmconfig:
clientConfig:
@@ -926,8 +889,6 @@ All versions less than v1.23.x are considered deprecated. Upgrade to a newer ver
You can reference Kubernetes in Terraform with the following code snippet.
-
-
@@ -999,7 +960,7 @@ data "spectrocloud_registry" "public_registry" {
}
data "spectrocloud_pack_simple" "k8s" {
- name = "kubernetes-gke"
+ name = "kubernetes-tke"
version = "1.24.4"
type = "helm"
registry_uid = data.spectrocloud_registry.public_registry.id
diff --git a/docs/docs-content/integrations/kubernetes.md b/docs/docs-content/integrations/kubernetes.md
index 1cad5142f5..37b2b294a2 100644
--- a/docs/docs-content/integrations/kubernetes.md
+++ b/docs/docs-content/integrations/kubernetes.md
@@ -28,16 +28,13 @@ A benefit of Palette when used with PXK is the ability to apply different flavor
There are no changes to the Kubernetes source code and we also follow the same versioning schema as the upstream open-source Kubernetes distribution.
-
-
-
:::info
We also offer Palette eXtended Kubernetes Edge (PXK-E) for Edge deployments. Refer to the [PXK-E glossary definition](/glossary-all#paletteextendedkubernetesedge(pxk-e)) to learn more about PXK-E.
:::
-# Versions Supported
+## Versions Supported
@@ -62,10 +59,10 @@ We also offer Palette eXtended Kubernetes Edge (PXK-E) for Edge deployments. Ref
| Parameter | Description |
|-----------|-------------|
-| `pack.palette.config.dashboard.identityProvider`| OIDC identity provider configuration. |
| `pack.podCIDR` | The CIDR range for Pods in the cluster. This should match the networking layer property. Default: `192.168.0.0/16`|
| `pack.serviceClusterIpRange` | The CIDR range for services in the cluster. This should not overlap with any IP ranges assigned to nodes or pods. Default: `10.96.0.0/12`|
-| ``pack.palette.config.dashboard.identityProvider`` | Dynamically enabled OpenID Connect (OIDC) Identity Provider (IDP) setting based on your UI selection when you add the PXK pack to your profile. This parameter appears in the YAML file after you make a selection. Refer to [Configure OIDC Identity Provider](/integrations/kubernetes#configureoidcidentityprovider). |
+| `pack.serviceDomain` | The cluster DNS service domain. Default: `cluster.local`. To change the default, you must add this parameter to the Kubernetes YAML file at cluster creation and specify the cluster DNS service domain to use. This value cannot be changed after cluster creation is complete. Refer to the [Change Cluster DNS Service Domain](/integrations/kubernetes?versions=k8s_v1.27#change-cluster-dns-service-domain) section. |
+| `pack.palette.config.dashboard.identityProvider` | Dynamically enabled OpenID Connect (OIDC) Identity Provider (IDP) setting based on your UI selection when you add the PXK pack to your profile. This parameter appears in the YAML file after you make a selection. Refer to [Configure OIDC Identity Provider](/integrations/kubernetes#configure-oidc-identity-provider). |
| `kubeadmconfig.apiServer.extraArgs` | A list of additional apiServer flags you can set.|
| `kubeadmconfig.apiServer.extraVolumes` | A list of additional volumes to mount on the apiServer.|
| `kubeadmconfig.controllerManager.extraArgs` | A list of additional ControllerManager flags to set.|
@@ -74,32 +71,45 @@ We also offer Palette eXtended Kubernetes Edge (PXK-E) for Edge deployments. Ref
| `kubeadmconfig.files` | A list of additional files to copy to the nodes.|
| `kubeadmconfig.preKubeadmCommands` | A list of additional commands to invoke **before** running kubeadm commands.|
| `kubeadmconfig.postKubeadmCommands` | A list of additional commands to invoke **after** running kubeadm commands.|
-| `kubeadmconfig.clientConfig` | Settings to manually configure OIDC-based authentication when you choose a third-party (Custom) IDP. Refer to [Configure Custom OIDC](/integrations/kubernetes#configurecustomoidc). |
-| `pack.serviceDomain` | The DNS name for the service domain in the cluster. Default: `cluster.local`.|
-
+| `kubeadmconfig.clientConfig` | Settings to manually configure OIDC-based authentication when you choose a third-party (Custom) IDP. Refer to [Configure Custom OIDC](/integrations/kubernetes#configure-custom-oidc). |
## Usage
The Kubeadm configuration file is where you can do the following:
-
-
- Change the default ``podCIDR`` and ``serviceClusterIpRange`` values. CIDR IPs specified in the configuration file take precedence over other defined CIDR IPs in your environment.
As you build your cluster, check that the ``podCIDR`` value does not overlap with any hosts or with the service network and the ``serviceClusterIpRange`` value does not overlap with any IP ranges assigned to nodes or pods. For more information, refer to the [Clusters](/clusters) guide and [Cluster Deployment Errors](https://docs.spectrocloud.com/troubleshooting/cluster-deployment).
+- Change the default cluster DNS service domain from `cluster.local` to a DNS domain that you specify. You can only change the DNS domain during cluster creation. For more information, refer to [Change Cluster DNS Service Domain](/integrations/kubernetes?versions=k8s_v1.27#change-cluster-dns-service-domain).
-- Manually configure a third-party OIDC IDP. For more information, check out [Configure Custom OIDC](/integrations/kubernetes#configurecustomoidc).
-
+- Manually configure a third-party OpenID Connect (OIDC) Identity Provider (IDP). For more information, check out [Configure Custom OIDC](/integrations/kubernetes#configure-custom-oidc).
- Add a certificate for the Spectro Proxy pack if you want to use a reverse proxy with a Kubernetes cluster. For more information, refer to the [Spectro Proxy](/integrations/frp) guide.
+### Change Cluster DNS Service Domain
-#### Configuration Changes
+The `pack.serviceDomain` parameter with default value `cluster.local` is not visible in the Kubernetes YAML file, and its value can only be changed at cluster creation. To change the value, you must add `serviceDomain: "cluster.local"` to the Kubernetes YAML file when you create a cluster, and specify the service domain you want to use.
-The PXK Kubeadm configuration is updated to dynamically enable OIDC based on your IDP selection by adding the ``identityProvider`` parameter.
+```yaml hideClipboard
+pack:
+ k8sHardening: True
+ podCIDR: "172.16.0.0/16"
+ serviceClusterIPRange: "10.96.0.0/12"
+ serviceDomain: ""
+```
-
+:::caution
+
+You can only specify the service domain at cluster creation. After cluster creation completes, you cannot update the value. Attempting to update it results in the error `serviceDomain update is forbidden for existing cluster`.
+
+:::
+
+For more information about networking configuration with DNS domains, refer to the Kubernetes [Networking](https://kubernetes.io/docs/reference/config-api/kubeadm-config.v1beta3/#kubeadm-k8s-io-v1beta3-Networking) API documentation.
+
+### Configuration Changes
+
+The PXK Kubeadm configuration is updated to dynamically enable OIDC based on your IDP selection by adding the ``identityProvider`` parameter.
```yaml
palette:
@@ -110,9 +120,9 @@ palette:
-#### Example Kubeadm Configuration File
+### Example Kubeadm Configuration File
-```yaml
+```yaml hideClipboard
pack:
k8sHardening: True
podCIDR: "192.168.0.0/16"
@@ -229,42 +239,36 @@ Platforms that use PXK can use the OIDC IDP feature, which offers the convenienc
When you add the PXK pack to a cluster profile, Palette displays the OIDC IDP options listed below.
-All the options require you to map a set of users or groups to a Kubernetes RBAC role. To learn how to map a Kubernetes role to users and groups, refer to [Create Role Bindings](/clusters/cluster-management/cluster-rbac/#createrolebindings).
-
-
-
+All the options require you to map a set of users or groups to a Kubernetes RBAC role. To learn how to map a Kubernetes role to users and groups, refer to [Create Role Bindings](/clusters/cluster-management/cluster-rbac#create-role-bindings).
- **None**: This setting does not require OIDC configuration for the cluster. It displays in the YAML file as `noauth`.
-
-
:::caution
We do not recommend choosing **None** in a production environment, as it may disable authentication for add-ons that rely on OIDC.
:::
-- **Custom**: This is the default setting and does not require OIDC configuration. However, if desired, it allows you to specify a third-party OIDC provider by configuring OIDC statements in the YAML file as described in [Configure Custom OIDC](/integrations/kubernetes#configurecustomoidc). This setting displays in the YAML file as `none`.
+
+- **Custom**: This is the default setting and does not require OIDC configuration. However, if desired, it allows you to specify a third-party OIDC provider by configuring OIDC statements in the YAML file as described in [Configure Custom OIDC](/integrations/kubernetes#configurecustomoidc). This setting displays in the YAML file as `none`.
- **Palette**: This setting makes Palette the IDP. Any user with a Palette account in the tenant and the proper permissions to view and access the project's resources is able to log into the Kubernetes dashboard. This setting displays in the YAML file as `palette`.
-
- **Inherit from Tenant**: This setting allows you to apply RBAC to multiple clusters and requires you to configure OpenID Connect (OIDC) in **Tenant Settings**. In Tenant Admin scope, navigate to **Tenant Settings** > **SSO**, choose **OIDC**, and provide your third-party IDP details. This setting displays in the YAML file as `tenant`. For more information, check out the [SSO Setup](/user-management/saml-sso) guide.
-:::info
+ :::info
-If your IDP uses Security Assertion Markup Language (SAML) authentication, then the **Inherit from Tenant** option will not work, and you will need to use the **Custom** option instead. This is because Kubernetes supports only OIDC authentication and not SAML authentication.
+ If your IDP uses Security Assertion Markup Language (SAML) authentication, then the **Inherit from Tenant** option will not work, and you will need to use the **Custom** option instead. This is because Kubernetes supports only OIDC authentication and not SAML authentication.
-:::
+ :::
+
### Configure Custom OIDC
The custom method to configure OIDC and apply RBAC for an OIDC provider can be used for all cloud services except Amazon Elastic Kubernetes Service (EKS) and [Azure-AKS](/clusters/public-cloud/azure/aks/#configureanazureactivedirectory).
-
-
@@ -272,32 +276,28 @@ The custom method to configure OIDC and apply RBAC for an OIDC provider can be u
Follow these steps to configure a third-party OIDC IDP. You can apply these steps to all the public cloud providers except Azure AKS and Amazon EKS clusters. Azure AKS and Amazon EKS require different configurations. AKS requires you to use Azure Active Directory (AAD) to enable OIDC integration. Refer to [Azure-AKS](/clusters/public-cloud/azure/aks/#configureanazureactivedirectory) to learn more. Click the **Amazon EKS** tab for steps to configure OIDC for EKS clusters.
-
-
1. Add the following parameters to your Kubernetes YAML file when creating a cluster profile.
-
-```yaml
-kubeadmconfig:
- apiServer:
- extraArgs:
- oidc-issuer-url: "provider URL"
- oidc-client-id: "client-id"
- oidc-groups-claim: "groups"
- oidc-username-claim: "email"
-```
+ ```yaml
+ kubeadmconfig:
+ apiServer:
+ extraArgs:
+ oidc-issuer-url: "provider URL"
+ oidc-client-id: "client-id"
+ oidc-groups-claim: "groups"
+ oidc-username-claim: "email"
+ ```
2. Under the `clientConfig` parameter section of Kubernetes YAML file, uncomment the `oidc-` configuration lines.
-
-```yaml
-kubeadmconfig:
- clientConfig:
- oidc-issuer-url: ""
- oidc-client-id: ""
- oidc-client-secret: ""
- oidc-extra-scope: profile,email,openid
-```
+ ```yaml
+ kubeadmconfig:
+ clientConfig:
+ oidc-issuer-url: ""
+ oidc-client-id: ""
+ oidc-client-secret: ""
+ oidc-extra-scope: profile,email,openid
+ ```
@@ -307,8 +307,6 @@ kubeadmconfig:
Follow these steps to configure OIDC for managed EKS clusters.
-
-
1. In the Kubernetes pack, uncomment the lines in the `oidcIdentityProvider` parameter section of the Kubernetes pack, and enter your third-party provider details.
```yaml
@@ -339,6 +337,7 @@ clientConfig:
+
### Use RBAC with OIDC
@@ -373,10 +372,10 @@ In this example, Palette is used as the IDP, and all users in the `dev-east-2` w
| Parameter | Description |
|-----------|-------------|
-| `pack.palette.config.dashboard.identityProvider`| OIDC identity provider configuration. |
| `pack.podCIDR` | The CIDR range for Pods in the cluster. This should match the networking layer property. Default: `192.168.0.0/16`|
| `pack.serviceClusterIpRange` | The CIDR range for services in the cluster. This should not overlap with any IP ranges assigned to nodes or pods. Default: `10.96.0.0/12`|
-| ``pack.palette.config.dashboard.identityProvider`` | Dynamically enabled OpenID Connect (OIDC) Identity Provider (IDP) setting based on your UI selection when you add the PXK pack to your profile. This parameter appears in the YAML file after you make a selection. Refer to [Configure OIDC Identity Provider](/integrations/kubernetes#configureoidcidentityprovider). |
+| `pack.serviceDomain` | The cluster DNS service domain. Default: `cluster.local`. To change the default, you must add this parameter to the Kubernetes YAML file at cluster creation and specify the cluster DNS service domain to use. This value cannot be changed after cluster creation is complete. Refer to the [Change Cluster DNS Service Domain](/integrations/kubernetes?versions=k8s_v1.26#change-cluster-dns-service-domain-1) section. |
+| `pack.palette.config.dashboard.identityProvider` | Dynamically enabled OpenID Connect (OIDC) Identity Provider (IDP) setting based on your UI selection when you add the PXK pack to your profile. This parameter appears in the YAML file after you make a selection. Refer to [Configure OIDC Identity Provider](/integrations/kubernetes#configureoidcidentityprovider). |
| `kubeadmconfig.apiServer.extraArgs` | A list of additional apiServer flags you can set.|
| `kubeadmconfig.apiServer.extraVolumes` | A list of additional volumes to mount on the apiServer.|
| `kubeadmconfig.controllerManager.extraArgs` | A list of additional ControllerManager flags to set.|
@@ -386,31 +385,46 @@ In this example, Palette is used as the IDP, and all users in the `dev-east-2` w
| `kubeadmconfig.preKubeadmCommands` | A list of additional commands to invoke **before** running kubeadm commands.|
| `kubeadmconfig.postKubeadmCommands` | A list of additional commands to invoke **after** running kubeadm commands.|
| `kubeadmconfig.clientConfig` | Settings to manually configure OIDC-based authentication when you choose a third-party (Custom) IDP. Refer to [Configure Custom OIDC](/integrations/kubernetes#configurecustomoidc). |
-| `pack.serviceDomain` | The DNS name for the service domain in the cluster. Default: `cluster.local`.|
-
## Usage
The Kubeadm configuration file is where you can do the following:
-
-
- Change the default ``podCIDR`` and ``serviceClusterIpRange`` values. CIDR IPs specified in the configuration file take precedence over other defined CIDR IPs in your environment.
As you build your cluster, check that the ``podCIDR`` value does not overlap with any hosts or with the service network and the ``serviceClusterIpRange`` value does not overlap with any IP ranges assigned to nodes or pods. For more information, refer to the [Clusters](/clusters) guide and [Cluster Deployment Errors](https://docs.spectrocloud.com/troubleshooting/cluster-deployment).
+- Manually configure a third-party OpenID Connect (OIDC) Identity Provider (IDP). For more information, check out [Configure Custom OIDC](/integrations/kubernetes#configurecustomoidc).
-- Manually configure a third-party OIDC IDP. For more information, check out [Configure Custom OIDC](/integrations/kubernetes#configurecustomoidc).
-
+- Change the default cluster DNS service domain from `cluster.local` to a DNS domain that you specify. You can only change the DNS domain during cluster creation. For more information, refer to [Change Cluster DNS Service Domain](/integrations/kubernetes?versions=k8s_v1.26#change-cluster-dns-service-domain-1).
- Add a certificate for the Spectro Proxy pack if you want to use a reverse proxy with a Kubernetes cluster. For more information, refer to the [Spectro Proxy](/integrations/frp) guide.
+### Change Cluster DNS Service Domain
+
+The `pack.serviceDomain` parameter with default value `cluster.local` is not visible in the Kubernetes YAML file, and its value can only be changed at cluster creation. To change the value, you must add `serviceDomain: "cluster.local"` to the Kubernetes YAML file when you create a cluster, and specify the service domain you want to use.
+
+```yaml
+pack:
+ k8sHardening: True
+ podCIDR: "172.16.0.0/16"
+ serviceClusterIPRange: "10.96.0.0/12"
+ serviceDomain: ""
+```
+
+:::caution
+
+You can only specify the service domain at cluster creation. After cluster creation completes, you cannot update the value. Attempting to update it results in the error `serviceDomain update is forbidden for existing cluster`.
+
+:::
+
+For more information about networking configuration with DNS domains, refer to the Kubernetes [Networking](https://kubernetes.io/docs/reference/config-api/kubeadm-config.v1beta3/#kubeadm-k8s-io-v1beta3-Networking) API documentation.
+
-#### Configuration Changes
+### Configuration Changes
The PXK Kubeadm configuration is updated to dynamically enable OIDC based on your IDP selection by adding the ``identityProvider`` parameter.
-
```yaml
palette:
@@ -421,7 +435,7 @@ palette:
-#### Example Kubeadm Configuration File
+### Example Kubeadm Configuration File
```yaml
pack:
@@ -543,39 +557,34 @@ When you add the PXK pack to a cluster profile, Palette displays the OIDC IDP op
All the options require you to map a set of users or groups to a Kubernetes RBAC role. To learn how to map a Kubernetes role to users and groups, refer to [Create Role Bindings](/clusters/cluster-management/cluster-rbac/#createrolebindings).
-
-
- **None**: This setting does not require OIDC configuration for the cluster. It displays in the YAML file as `noauth`.
-
-
:::caution
We do not recommend choosing **None** in a production environment, as it may disable authentication for add-ons that rely on OIDC.
:::
-- **Custom**: This is the default setting and does not require OIDC configuration. However, if desired, it allows you to specify a third-party OIDC provider by configuring OIDC statements in the YAML file as described in [Configure Custom OIDC](/integrations/kubernetes#configurecustomoidc). This setting displays in the YAML file as `none`.
+
+- **Custom**: This is the default setting and does not require OIDC configuration. However, if desired, it allows you to specify a third-party OIDC provider by configuring OIDC statements in the YAML file as described in [Configure Custom OIDC](/integrations/kubernetes#configurecustomoidc). This setting displays in the YAML file as `none`.
- **Palette**: This setting makes Palette the IDP. Any user with a Palette account in the tenant and the proper permissions to view and access the project's resources is able to log into the Kubernetes dashboard. This setting displays in the YAML file as `palette`.
-
- **Inherit from Tenant**: This setting allows you to apply RBAC to multiple clusters and requires you to configure OpenID Connect (OIDC) in **Tenant Settings**. In Tenant Admin scope, navigate to **Tenant Settings** > **SSO**, choose **OIDC**, and provide your third-party IDP details. This setting displays in the YAML file as `tenant`. For more information, check out the [SSO Setup](/user-management/saml-sso) guide.
-:::info
+ :::info
-If your IDP uses Security Assertion Markup Language (SAML) authentication, then the **Inherit from Tenant** option will not work, and you will need to use the **Custom** option instead. This is because Kubernetes supports only OIDC authentication and not SAML authentication.
+ If your IDP uses Security Assertion Markup Language (SAML) authentication, then the **Inherit from Tenant** option will not work, and you will need to use the **Custom** option instead. This is because Kubernetes supports only OIDC authentication and not SAML authentication.
-:::
+ :::
+
### Configure Custom OIDC
The custom method to configure OIDC and apply RBAC for an OIDC provider can be used for all cloud services except Amazon Elastic Kubernetes Service (EKS) and [Azure-AKS](/clusters/public-cloud/azure/aks/#configureanazureactivedirectory).
-
-
@@ -583,32 +592,28 @@ The custom method to configure OIDC and apply RBAC for an OIDC provider can be u
Follow these steps to configure a third-party OIDC IDP. You can apply these steps to all the public cloud providers except Azure AKS and Amazon EKS clusters. Azure AKS and Amazon EKS require different configurations. AKS requires you to use Azure Active Directory (AAD) to enable OIDC integration. Refer to [Azure-AKS](/clusters/public-cloud/azure/aks/#configureanazureactivedirectory) to learn more. Click the **Amazon EKS** tab for steps to configure OIDC for EKS clusters.
-
-
1. Add the following parameters to your Kubernetes YAML file when creating a cluster profile.
-
-```yaml
-kubeadmconfig:
- apiServer:
- extraArgs:
- oidc-issuer-url: "provider URL"
- oidc-client-id: "client-id"
- oidc-groups-claim: "groups"
- oidc-username-claim: "email"
-```
+ ```yaml
+ kubeadmconfig:
+ apiServer:
+ extraArgs:
+ oidc-issuer-url: "provider URL"
+ oidc-client-id: "client-id"
+ oidc-groups-claim: "groups"
+ oidc-username-claim: "email"
+ ```
2. Under the `clientConfig` parameter section of Kubernetes YAML file, uncomment the `oidc-` configuration lines.
-
-```yaml
-kubeadmconfig:
- clientConfig:
- oidc-issuer-url: ""
- oidc-client-id: ""
- oidc-client-secret: ""
- oidc-extra-scope: profile,email,openid
-```
+ ```yaml
+ ubeadmconfig:
+ clientConfig:
+ oidc-issuer-url: ""
+ oidc-client-id: ""
+ oidc-client-secret: ""
+ oidc-extra-scope: profile,email,openid
+ ```
@@ -618,31 +623,29 @@ kubeadmconfig:
Follow these steps to configure OIDC for managed EKS clusters.
-
-
1. In the Kubernetes pack, uncomment the lines in the `oidcIdentityProvider` parameter section of the Kubernetes pack, and enter your third-party provider details.
-```yaml
-oidcIdentityProvider:
- identityProviderConfigName: 'Spectro-docs'
- issuerUrl: 'issuer-url'
- clientId: 'user-client-id-from-Palette'
- usernameClaim: "email"
- usernamePrefix: "-"
- groupsClaim: "groups"
- groupsPrefix: ""
- requiredClaims:
-```
+ ```yaml hideClipboard
+ oidcIdentityProvider:
+ identityProviderConfigName: 'Spectro-docs'
+ issuerUrl: 'issuer-url'
+ clientId: 'user-client-id-from-Palette'
+ usernameClaim: "email"
+ usernamePrefix: "-"
+ groupsClaim: "groups"
+ groupsPrefix: ""
+ requiredClaims:
+ ```
2. Under the `clientConfig` parameter section of Kubernetes pack, uncomment the `oidc-` configuration lines.
-```yaml
-clientConfig:
- oidc-issuer-url: "{{ .spectro.pack.kubernetes-eks.managedControlPlane.oidcIdentityProvider.issuerUrl }}"
- oidc-client-id: "{{ .spectro.pack.kubernetes-eks.managedControlPlane.oidcIdentityProvider.clientId }}"
- oidc-client-secret: yourSecretClientSecretGoesHere
- oidc-extra-scope: profile,email
-```
+ ```yaml hideClipboard
+ clientConfig:
+ oidc-issuer-url: "{{ .spectro.pack.kubernetes-eks.managedControlPlane.oidcIdentityProvider.issuerUrl }}"
+ oidc-client-id: "{{ .spectro.pack.kubernetes-eks.managedControlPlane.oidcIdentityProvider.clientId }}"
+ oidc-client-secret: yourSecretClientSecretGoesHere
+ oidc-extra-scope: profile,email
+ ```
3. Provide third-party OIDC IDP details.
@@ -650,6 +653,7 @@ clientConfig:
+
### Use RBAC with OIDC
@@ -686,44 +690,63 @@ In this example, Palette is used as the IDP, and all users in the `dev-east-2` w
| Parameter | Description |
|-----------|-------------|
-| ``pack.podCIDR`` | The CIDR range for Pods in the cluster. This should match the networking layer property. Default: `192.168.0.0/16`|
-| ``pack.serviceClusterIpRange`` | The CIDR range for services in the cluster. This should not overlap with any IP ranges assigned to nodes or pods. Default: `10.96.0.0/12`|
-| ``pack.palette.config.dashboard.identityProvider`` | Dynamically enabled OpenID Connect (OIDC) Identity Provider (IDP) setting based on your UI selection when you add the PXK pack to your profile. This parameter appears in the YAML file after you make a selection. Refer to [Configure OIDC Identity Provider](/integrations/kubernetes#configureoidcidentityprovider). |
-| ``kubeadmconfig.apiServer.extraArgs`` | A list of additional apiServer flags you can set.|
-| ``kubeadmconfig.apiServer.extraVolumes`` | A list of additional volumes to mount on the apiServer.|
-| ``kubeadmconfig.controllerManager.extraArgs`` | A list of additional ControllerManager flags to set.|
-| ``kubeadmconfig.scheduler.extraArgs`` | A list of additional Kube scheduler flags to set.|
-| ``kubeadmconfig.kubeletExtraArgs`` | A list of kubelet arguments to set and copy to the nodes.|
-| ``kubeadmconfig.files`` | A list of additional files to copy to the nodes.|
-| ``kubeadmconfig.preKubeadmCommands`` | A list of additional commands to invoke **before** running kubeadm commands.|
-| ``kubeadmconfig.postKubeadmCommands`` | A list of additional commands to invoke **after** running kubeadm commands.|
-| ``kubeadmconfig.clientConfig`` | Settings to manually configure OIDC-based authentication when you choose a third-party (Custom) IDP. Refer to [Configure Custom OIDC](/integrations/kubernetes#configurecustomoidc). |
-| ``pack.serviceDomain`` | The DNS name for the service domain in the cluster. Default: ``cluster.local``.|
+| `pack.podCIDR` | The CIDR range for Pods in the cluster. This should match the networking layer property. Default: `192.168.0.0/16`|
+| `pack.serviceClusterIpRange` | The CIDR range for services in the cluster. This should not overlap with any IP ranges assigned to nodes or pods. Default: `10.96.0.0/12`|
+| `pack.serviceDomain` | The cluster DNS service domain. Default: `cluster.local`. To change the default, you must add this parameter to the Kubernetes YAML file at cluster creation and specify the cluster DNS service domain to use. This value cannot be changed after cluster creation is complete. Refer to the [Change Cluster DNS Service Domain](/integrations/kubernetes?versions=k8s_v1.25#change-cluster-dns-service-domain) section. |
+| `pack.palette.config.dashboard.identityProvider` | Dynamically enabled OpenID Connect (OIDC) Identity Provider (IDP) setting based on your UI selection when you add the PXK pack to your profile. This parameter appears in the YAML file after you make a selection. Refer to [Configure OIDC Identity Provider](/integrations/kubernetes#configureoidcidentityprovider). |
+| `kubeadmconfig.apiServer.extraArgs` | A list of additional apiServer flags you can set.|
+| `kubeadmconfig.apiServer.extraVolumes` | A list of additional volumes to mount on the apiServer.|
+| `kubeadmconfig.controllerManager.extraArgs` | A list of additional ControllerManager flags to set.|
+| `kubeadmconfig.scheduler.extraArgs` | A list of additional Kube scheduler flags to set.|
+| `kubeadmconfig.kubeletExtraArgs` | A list of kubelet arguments to set and copy to the nodes.|
+| `kubeadmconfig.files` | A list of additional files to copy to the nodes.|
+| `kubeadmconfig.preKubeadmCommands` | A list of additional commands to invoke **before** running kubeadm commands.|
+| `kubeadmconfig.postKubeadmCommands` | A list of additional commands to invoke **after** running kubeadm commands.|
+| `kubeadmconfig.clientConfig` | Settings to manually configure OIDC-based authentication when you choose a third-party (Custom) IDP. Refer to [Configure Custom OIDC](/integrations/kubernetes#configurecustomoidc). |
## Usage
The Kubeadm configuration file is where you can do the following:
-
-
- Change the default ``podCIDR`` and ``serviceClusterIpRange`` values. CIDR IPs specified in the configuration file take precedence over other defined CIDR IPs in your environment.
As you build your cluster, check that the ``podCIDR`` value does not overlap with any hosts or with the service network and the ``serviceClusterIpRange`` value does not overlap with any IP ranges assigned to nodes or pods. For more information, refer to the [Clusters](/clusters) guide and [Cluster Deployment Errors](https://docs.spectrocloud.com/troubleshooting/cluster-deployment).
+- Manually configure a third-party OpenID Connect (OIDC) Identity Provider (IDP). For more information, check out [Configure Custom OIDC](/integrations/kubernetes#configurecustomoidc).
-- Manually configure a third-party OIDC IDP. For more information, check out [Configure Custom OIDC](/integrations/kubernetes#configurecustomoidc).
-
+- Change the default cluster DNS service domain from `cluster.local` to a DNS domain that you specify. You can only change the DNS domain during cluster creation. For more information, refer to [Change Cluster DNS Service Domain](/integrations/kubernetes?versions=k8s_v1.25#change-cluster-dns-service-domain).
- Add a certificate for the Spectro Proxy pack if you want to use a reverse proxy with a Kubernetes cluster. For more information, refer to the [Spectro Proxy](/integrations/frp) guide.
+### Change Cluster DNS Service Domain
+
+The `pack.serviceDomain` parameter with default value `cluster.local` is not visible in the Kubernetes YAML file, and its value can only be changed at cluster creation. To change the value, you must add `serviceDomain: "cluster.local"` to the Kubernetes YAML file when you create a cluster, and specify the service domain you want to use.
-#### Configuration Changes
+```yaml
+pack:
+ k8sHardening: True
+ podCIDR: "172.16.0.0/16"
+ serviceClusterIPRange: "10.96.0.0/12"
+ serviceDomain: ""
+```
+
+:::caution
+
+You can only specify the service domain at cluster creation. After cluster creation completes, you cannot update the value. Attempting to update it results in the error `serviceDomain update is forbidden for existing cluster`.
+
+:::
+
+For more information about networking configuration with DNS domains, refer to the Kubernetes [Networking](https://kubernetes.io/docs/reference/config-api/kubeadm-config.v1beta3/#kubeadm-k8s-io-v1beta3-Networking) API documentation.
+
+
+
+### Configuration Changes
The PXK Kubeadm configuration is updated to dynamically enable OIDC based on your IDP selection by adding the ``identityProvider`` parameter.
-```yaml
+```yaml hideClipboard
palette:
config:
dashboard:
@@ -732,9 +755,9 @@ palette:
-#### Example Kubeadm Configuration File
+### Example Kubeadm Configuration File
-```yaml
+```yaml hideClipboard
pack:
k8sHardening: True
podCIDR: "192.168.0.0/16"
@@ -852,38 +875,34 @@ When you add the PXK pack to a cluster profile, Palette displays the OIDC IDP op
All the options require you to map a set of users or groups to a Kubernetes RBAC role. To learn how to map a Kubernetes role to users and groups, refer to [Create Role Bindings](/clusters/cluster-management/cluster-rbac/#createrolebindings).
-
-
- **None**: This is the default setting and there is nothing to configure. This setting displays in the YAML file as `noauth`.
-
-
:::caution
We do not recommend choosing **None** in a production environment, as it may disable authentication for add-ons that rely on OIDC.
:::
-- **Custom**: This setting allows you to specify a third-party OIDC provider by configuring OIDC statements in the Kubeadm configuration file as described in [Configure Custom OIDC](/integrations/kubernetes#configurecustomoidc). This setting displays in the YAML file as `none`.
+
+- **Custom**: This setting allows you to specify a third-party OIDC provider by configuring OIDC statements in the Kubeadm configuration file as described in [Configure Custom OIDC](/integrations/kubernetes#configurecustomoidc). This setting displays in the YAML file as `none`.
- **Palette**: This setting makes Palette the IDP. Any user with a Palette account in the tenant and the proper permissions to view and access the project's resources is able to log into the Kubernetes dashboard. This setting displays in the YAML file as `palette`.
-
- **Inherit from Tenant**: This setting allows you to apply RBAC to multiple clusters and requires you to configure OpenID Connect (OIDC) in **Tenant Settings**. In Tenant Admin scope, navigate to **Tenant Settings** > **SSO**, choose **OIDC**, and provide your third-party IDP details. This setting displays in the YAML file as `tenant`. For more information, check out the [SSO Setup](/user-management/saml-sso) guide.
-:::info
+ :::info
-If your IDP uses Security Assertion Markup Language (SAML) authentication, then the **Inherit from Tenant** option will not work, and you will need to use the **Custom** option instead. This is because Kubernetes supports only OIDC authentication and not SAML authentication.
+ If your IDP uses Security Assertion Markup Language (SAML) authentication, then the **Inherit from Tenant** option will not work, and you will need to use the **Custom** option instead. This is because Kubernetes supports only OIDC authentication and not SAML authentication.
-:::
+ :::
+
+
### Configure Custom OIDC
The custom method to configure OIDC and apply RBAC for an OIDC provider can be used for all cloud services except Amazon Elastic Kubernetes Service (EKS) and [Azure-AKS](/clusters/public-cloud/azure/aks/#configureanazureactivedirectory).
-
-
@@ -891,32 +910,28 @@ The custom method to configure OIDC and apply RBAC for an OIDC provider can be u
Follow these steps to configure a third-party OIDC IDP. You can apply these steps to all the public cloud providers except Azure AKS and Amazon EKS clusters. Azure AKS and Amazon EKS require different configurations. AKS requires you to use Azure Active Directory (AAD) to enable OIDC integration. Refer to [Azure-AKS](/clusters/public-cloud/azure/aks/#configureanazureactivedirectory) to learn more. Click the **Amazon EKS** tab for steps to configure OIDC for EKS clusters.
-
-
1. Add the following parameters to your Kubernetes YAML file when creating a cluster profile.
-
-```yaml
-kubeadmconfig:
- apiServer:
- extraArgs:
- oidc-issuer-url: "provider URL"
- oidc-client-id: "client-id"
- oidc-groups-claim: "groups"
- oidc-username-claim: "email"
-```
+ ```yaml hideClipboard
+ kubeadmconfig:
+ apiServer:
+ extraArgs:
+ oidc-issuer-url: "provider URL"
+ oidc-client-id: "client-id"
+ oidc-groups-claim: "groups"
+ oidc-username-claim: "email"
+ ```
2. Under the `clientConfig` parameter section of Kubernetes YAML file, uncomment the `oidc-` configuration lines.
-
-```yaml
-kubeadmconfig:
- clientConfig:
- oidc-issuer-url: ""
- oidc-client-id: ""
- oidc-client-secret: ""
- oidc-extra-scope: profile,email,openid
-```
+ ```yaml hideClipboard
+ kubeadmconfig:
+ clientConfig:
+ oidc-issuer-url: ""
+ oidc-client-id: ""
+ oidc-client-secret: ""
+ oidc-extra-scope: profile,email,openid
+ ```
3. Provide third-party OIDC IDP details. Refer to the [SAML & SSO Setup](/user-management/saml-sso) for guidance on configuring a third party IDP with Palette.
@@ -932,16 +947,16 @@ Follow these steps to configure OIDC for managed EKS clusters.
1. In the Kubernetes pack, uncomment the lines in the `oidcIdentityProvider` parameter section of the Kubernetes pack, and enter your third-party provider details.
```yaml
-oidcIdentityProvider:
- identityProviderConfigName: 'Spectro-docs'
- issuerUrl: 'issuer-url'
- clientId: 'user-client-id-from-Palette'
- usernameClaim: "email"
- usernamePrefix: "-"
- groupsClaim: "groups"
- groupsPrefix: ""
- requiredClaims:
-```
+ oidcIdentityProvider: hideClipboard
+ identityProviderConfigName: 'Spectro-docs'
+ issuerUrl: 'issuer-url'
+ clientId: 'user-client-id-from-Palette'
+ usernameClaim: "email"
+ usernamePrefix: "-"
+ groupsClaim: "groups"
+ groupsPrefix: ""
+ requiredClaims:
+ ```
2. Under the `clientConfig` parameter section of Kubernetes pack, uncomment the `oidc-` configuration lines.
@@ -957,6 +972,8 @@ clientConfig:
+
+
### Use RBAC with OIDC
You can create a role binding that uses individual users as the subject or specify a group name as the subject to map many users to a role. The group name is the group assigned in the OIDC provider's configuration. Below is an example. To learn more, review [Create Role Bindings](/clusters/cluster-management/cluster-rbac/#createrolebindings).
@@ -994,6 +1011,7 @@ In this example, Palette is used as the IDP, and all users in the `dev-east-2` w
|-----------|-------------|
| ``pack.podCIDR`` | The CIDR range for Pods in cluster. This should match the networking layer property. Default: `192.168.0.0/16`|
| ``pack.serviceClusterIpRange`` | The CIDR range for services in the cluster. This should not overlap with any IP ranges assigned to nodes or pods. Default: `10.96.0.0/12`|
+| `pack.serviceDomain` | The cluster DNS service domain. Default: `cluster.local`. To change the default, you must add this parameter to the Kubernetes YAML file at cluster creation and specify the cluster DNS service domain to use. This value cannot be changed after cluster creation is complete. Refer to the [Change Cluster DNS Service Domain](/integrations/kubernetes?versions=k8s_v1.24#change-cluster-dns-service-domain) section. |
| ``kubeadmconfig.apiServer.extraArgs`` | A list of additional apiServer flags you can set.|
| ``kubeadmconfig.apiServer.extraVolumes`` | A list of additional volumes to mount on apiServer.|
| ``kubeadmconfig.controllerManager.extraArgs`` | A list of additional ControllerManager flags to set.|
@@ -1002,32 +1020,47 @@ In this example, Palette is used as the IDP, and all users in the `dev-east-2` w
| ``kubeadmconfig.files`` | A list of additional files to copy to the nodes. |
| ``kubeadmconfig.preKubeadmCommands`` | A list of additional commands to invoke **before** running kubeadm commands.|
| ``kubeadmconfig.postKubeadmCommands`` | A list of additional commands to invoke **after** running kubeadm commands.|
-| ``pack.serviceDomain`` | The DNS name for the service domain in the cluster. Default: ``cluster.local``.|
-
## Usage
The Kubeadm configuration file is where you can do the following:
-
-
- Change the default ``podCIDR`` and ``serviceClusterIpRange`` values. CIDR IPs specified in the configuration file take precedence over other defined CIDR IPs in your environment.
As you build your cluster, check that the ``podCIDR`` value does not overlap with any hosts or with the service network and the ``serviceClusterIpRange`` value does not overlap with any IP ranges assigned to nodes or pods. For more information, refer to the [Clusters](/clusters) guide and [Cluster Deployment Errors](https://docs.spectrocloud.com/troubleshooting/cluster-deployment).
+- Manually configure a third-party OpenID Connect (OIDC) Identity Provider (IDP). For more information, check out [Configure Custom OIDC](/integrations/kubernetes#configurecustomoidc).
-- Manually configure a third-party OIDC IDP. For more information, check out [Configure Custom OIDC](/integrations/kubernetes#configurecustomoidc).
-
+- Change the default cluster DNS service domain from `cluster.local` to a DNS domain that you specify. You can only change the DNS domain during cluster creation. For more information, refer to [Change Cluster DNS Service Domain](/integrations/kubernetes?versions=k8s_v1.24#change-cluster-dns-service-domain).
- Add a certificate for the Spectro Proxy pack if you want to use a reverse proxy with a Kubernetes cluster. For more information, refer to the [Spectro Proxy](/integrations/frp) guide.
+### Change Cluster DNS Service Domain
-#### Configuration Changes
+The `pack.serviceDomain` parameter with default value `cluster.local` is not visible in the Kubernetes YAML file, and its value can only be changed at cluster creation. To change the value, you must add `serviceDomain: "cluster.local"` to the Kubernetes YAML file when you create a cluster, and specify the service domain you want to use.
-The PXK Kubeadm configuration is updated to dynamically enable OIDC based on your IDP selection by adding the ``identityProvider`` parameter.
+```yaml
+pack:
+ k8sHardening: True
+ podCIDR: "172.16.0.0/16"
+ serviceClusterIPRange: "10.96.0.0/12"
+ serviceDomain: ""
+```
+
+:::caution
+
+You can only specify the service domain at cluster creation. After cluster creation completes, you cannot update the value. Attempting to update it results in the error `serviceDomain update is forbidden for existing cluster`.
+
+:::
+
+For more information about networking configuration with DNS domains, refer to the Kubernetes [Networking](https://kubernetes.io/docs/reference/config-api/kubeadm-config.v1beta3/#kubeadm-k8s-io-v1beta3-Networking) API documentation.
+### Configuration Changes
+
+The PXK Kubeadm configuration is updated to dynamically enable OIDC based on your IDP selection by adding the ``identityProvider`` parameter.
+
```yaml
palette:
config:
@@ -1037,7 +1070,7 @@ palette:
-#### Example Kubeadm Configuration File
+### Example Kubeadm Configuration File
```yaml
pack:
@@ -1126,39 +1159,34 @@ When you add the PXK pack to a cluster profile, Palette displays the OIDC IDP op
All the options require you to map a set of users or groups to a Kubernetes RBAC role. To learn how to map a Kubernetes role to users and groups, refer to [Create Role Bindings](/clusters/cluster-management/cluster-rbac/#createrolebindings).
-
-
- **None**: This is the default setting and there is nothing to configure. This setting displays in the YAML file as `noauth`.
-
-
:::caution
We do not recommend choosing **None** in a production environment, as it may disable authentication for add-ons that rely on OIDC.
:::
-- **Custom**: This setting allows you to specify a third-party OIDC provider by configuring OIDC statements in the Kubeadm configuration file as described in [Configure Custom OIDC](/integrations/kubernetes#configurecustomoidc). This setting displays in the YAML file as `none`.
+
+- **Custom**: This setting allows you to specify a third-party OIDC provider by configuring OIDC statements in the Kubeadm configuration file as described in [Configure Custom OIDC](/integrations/kubernetes#configurecustomoidc). This setting displays in the YAML file as `none`.
- **Palette**: This setting makes Palette the IDP. Any user with a Palette account in the tenant and the proper permissions to view and access the project's resources is able to log into the Kubernetes dashboard. This setting displays in the YAML file as `palette`.
-
- **Inherit from Tenant**: This setting allows you to apply RBAC to multiple clusters and requires you to configure OpenID Connect (OIDC) in **Tenant Settings**. In Tenant Admin scope, navigate to **Tenant Settings** > **SSO**, choose **OIDC**, and provide your third-party IDP details. This setting displays in the YAML file as `tenant`. For more information, check out the [SSO Setup](/user-management/saml-sso) guide.
-:::info
+ :::info
-If your IDP uses Security Assertion Markup Language (SAML) authentication, then the **Inherit from Tenant** option will not work, and you will need to use the **Custom** option instead. This is because Kubernetes supports only OIDC authentication and not SAML authentication.
+ If your IDP uses Security Assertion Markup Language (SAML) authentication, then the **Inherit from Tenant** option will not work, and you will need to use the **Custom** option instead. This is because Kubernetes supports only OIDC authentication and not SAML authentication.
-:::
+ :::
+
### Configure Custom OIDC
The custom method to configure OIDC and apply RBAC for an OIDC provider can be used for all cloud services except Amazon Elastic Kubernetes Service (EKS) and [Azure-AKS](/clusters/public-cloud/azure/aks/#configureanazureactivedirectory).
-
-
@@ -1166,32 +1194,28 @@ The custom method to configure OIDC and apply RBAC for an OIDC provider can be u
Follow these steps to configure a third-party OIDC IDP. You can apply these steps to all the public cloud providers except Azure AKS and Amazon EKS clusters. Azure AKS and Amazon EKS require different configurations. AKS requires you to use Azure Active Directory (AAD) to enable OIDC integration. Refer to [Azure-AKS](/clusters/public-cloud/azure/aks/#configureanazureactivedirectory) to learn more. Click the **Amazon EKS** tab for steps to configure OIDC for EKS clusters.
-
-
1. Add the following parameters to your Kubernetes YAML file when creating a cluster profile.
-
-```yaml
-kubeadmconfig:
- apiServer:
- extraArgs:
- oidc-issuer-url: "provider URL"
- oidc-client-id: "client-id"
- oidc-groups-claim: "groups"
- oidc-username-claim: "email"
-```
+ ```yaml
+ kubeadmconfig:
+ apiServer:
+ extraArgs:
+ oidc-issuer-url: "provider URL"
+ oidc-client-id: "client-id"
+ oidc-groups-claim: "groups"
+ oidc-username-claim: "email"
+ ```
2. Under the `clientConfig` parameter section of Kubernetes YAML file, uncomment the `oidc-` configuration lines.
-
-```yaml
-kubeadmconfig:
- clientConfig:
- oidc-issuer-url: ""
- oidc-client-id: ""
- oidc-client-secret: ""
- oidc-extra-scope: profile,email,openid
-```
+ ```yaml
+ kubeadmconfig:
+ clientConfig:
+ oidc-issuer-url: ""
+ oidc-client-id: ""
+ oidc-client-secret: ""
+ oidc-extra-scope: profile,email,openid
+ ```
3. Provide third-party OIDC IDP details.
@@ -1206,32 +1230,34 @@ Follow these steps to configure OIDC for managed EKS clusters.
1. In the Kubernetes pack, uncomment the lines in the `oidcIdentityProvider` parameter section of the Kubernetes pack, and enter your third-party provider details.
-```yaml
-oidcIdentityProvider:
- identityProviderConfigName: 'Spectro-docs'
- issuerUrl: 'issuer-url'
- clientId: 'user-client-id-from-Palette'
- usernameClaim: "email"
- usernamePrefix: "-"
- groupsClaim: "groups"
- groupsPrefix: ""
- requiredClaims:
-```
+ ```yaml
+ oidcIdentityProvider:
+ identityProviderConfigName: 'Spectro-docs'
+ issuerUrl: 'issuer-url'
+ clientId: 'user-client-id-from-Palette'
+ usernameClaim: "email"
+ usernamePrefix: "-"
+ groupsClaim: "groups"
+ groupsPrefix: ""
+ requiredClaims:
+ ```
2. Under the `clientConfig` parameter section of Kubernetes pack, uncomment the `oidc-` configuration lines.
-```yaml
-clientConfig:
- oidc-issuer-url: "{{ .spectro.pack.kubernetes-eks.managedControlPlane.oidcIdentityProvider.issuerUrl }}"
- oidc-client-id: "{{ .spectro.pack.kubernetes-eks.managedControlPlane.oidcIdentityProvider.clientId }}"
- oidc-client-secret: yourSecretClientSecretGoesHere
- oidc-extra-scope: profile,email
-```
+ ```yaml
+ clientConfig:
+ oidc-issuer-url: "{{ .spectro.pack.kubernetes-eks.managedControlPlane.oidcIdentityProvider.issuerUrl }}"
+ oidc-client-id: "{{ .spectro.pack.kubernetes-eks.managedControlPlane.oidcIdentityProvider.clientId }}"
+ oidc-client-secret: yourSecretClientSecretGoesHere
+ oidc-extra-scope: profile,email
+ ```
+
+
### Use RBAC with OIDC
You can create a role binding that uses individual users as the subject or specify a group name as the subject to map many users to a role. The group name is the group assigned in the OIDC provider's configuration. Below is an example. To learn more, review [Create Role Bindings](/clusters/cluster-management/cluster-rbac/#createrolebindings).
@@ -1264,8 +1290,6 @@ All versions less than v1.23.x are considered deprecated. Upgrade to a newer ver
You can reference Kubernetes in Terraform with the following code snippet.
-
-
```hcl
data "spectrocloud_registry" "public_registry" {
name = "Public Repo"