Skip to content

Commit

Permalink
settings: adds new node-labels, node-taints settings
Browse files Browse the repository at this point in the history
Adds `node-labels` and `node-taints` settings for kubelet.
Allows users to label and taint the node through userdata

Updates README with descriptions of the new settings
  • Loading branch information
etungsten committed Oct 15, 2019
1 parent 288d55e commit d2e60fe
Show file tree
Hide file tree
Showing 4 changed files with 23 additions and 0 deletions.
13 changes: 13 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -237,6 +237,19 @@ See the [setup guide](INSTALL.md) for *much* more detail on setting up Thar and
* `settings.kubernetes.cluster-certificate`: This is the base64-encoded certificate authority of the cluster.
* `settings.kubernetes.api-server`: This is the cluster's Kubernetes API endpoint.

The following settings can be optionally set to customize the node labels and taints.
* `settings.kubernetes.node-labels`: [Labels](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/) in the form of key, value pairs added when registering the node in the cluster.
* `settings.kubernetes.node-taints`: [Taints](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/) in the form of key, value and effect entries added when registering the node in the cluster.
* Example user data for setting up labels and taints:
```
[settings.kubernetes.node-labels]
label1 = "foo"
label2 = "bar"
[settings.kubernetes.node-taints]
dedicated = "experimental:PreferNoSchedule"
special = "true:NoSchedule"
```

The following settings are set for you automatically by [pluto](workspaces/api/) based on runtime instance information, but you can override them if you know what you're doing!
* `settings.kubernetes.max-pods`: The maximum number of pods that can be scheduled on this node (limited by number of available IPv4 addresses)
* `settings.kubernetes.cluster-dns-ip`: The CIDR block of the primary network interface.
Expand Down
2 changes: 2 additions & 0 deletions packages/kubernetes/kubelet-env
Original file line number Diff line number Diff line change
@@ -1,2 +1,4 @@
NODE_IP={{settings.kubernetes.node-ip}}
NODE_LABELS={{join_map "=" "," "no-fail-if-missing" settings.kubernetes.node-labels}}
NODE_TAINTS={{join_map "=" "," "no-fail-if-missing" settings.kubernetes.node-taints}}
POD_INFRA_CONTAINER_IMAGE={{settings.kubernetes.pod-infra-container-image}}
2 changes: 2 additions & 0 deletions packages/kubernetes/kubelet.service
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@ ExecStart=/usr/bin/kubelet \
--cert-dir /var/lib/kubelet/pki \
--volume-plugin-dir /var/lib/kubelet/plugins/volume/exec \
--node-ip ${NODE_IP} \
--node-labels "${NODE_LABELS}" \
--register-with-taints "${NODE_TAINTS}" \
--pod-infra-container-image ${POD_INFRA_CONTAINER_IMAGE}

Restart=on-failure
Expand Down
6 changes: 6 additions & 0 deletions workspaces/api/apiserver/src/model.rs
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,12 @@ pub struct KubernetesSettings {
#[serde(skip_serializing_if = "Option::is_none")]
pub api_server: Option<String>,

#[serde(skip_serializing_if = "Option::is_none")]
pub node_labels: Option<HashMap<SingleLineString, SingleLineString>>,

#[serde(skip_serializing_if = "Option::is_none")]
pub node_taints: Option<HashMap<SingleLineString, SingleLineString>>,

// Dynamic settings.

#[serde(skip_serializing_if = "Option::is_none")]
Expand Down

0 comments on commit d2e60fe

Please sign in to comment.