Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change default kube-system DaemonSet tolerations #682

Merged
merged 1 commit into from
Mar 31, 2020

Conversation

dghubble
Copy link
Member

@dghubble dghubble commented Mar 27, 2020

  • Change kube-proxy, flannel, and calico-node DaemonSet tolerations to tolerate node.kubernetes.io/not-ready and node-role.kubernetes.io/master (i.e. controllers) explicitly, rather than tolerating all taints
  • kube-system DaemonSets will no longer tolerate custom node taints by default. Instead, custom node taints must be enumerated to opt-in to scheduling/executing the kube-system DaemonSets
  • Consider setting the daemonset_tolerations variable of terraform-render-bootstrap at a later date

Background: Tolerating all taints ruled out use-cases where certain nodes might legitimately need to keep kube-proxy or CNI networking disabled
Related: poseidon/terraform-render-bootstrap#179

@dghubble dghubble force-pushed the use-explicit-daemonset-tolerations branch 2 times, most recently from c621836 to a2d4146 Compare March 31, 2020 07:59
* Change kube-proxy, flannel, and calico-node DaemonSet
tolerations to tolerate `node.kubernetes.io/not-ready`
and `node-role.kubernetes.io/master` (i.e. controllers)
explicitly, rather than tolerating all taints
* kube-system DaemonSets will no longer tolerate custom
node taints by default. Instead, custom node taints must
be enumerated to opt-in to scheduling/executing the
kube-system DaemonSets
* Consider setting the daemonset_tolerations variable
of terraform-render-bootstrap at a later date

Background: Tolerating all taints ruled out use-cases
where certain nodes might legitimately need to keep
kube-proxy or CNI networking disabled
Related: poseidon/terraform-render-bootstrap#179
@dghubble dghubble force-pushed the use-explicit-daemonset-tolerations branch from a2d4146 to bac5acb Compare March 31, 2020 08:01
@dghubble dghubble merged commit bac5acb into master Mar 31, 2020
@dghubble dghubble deleted the use-explicit-daemonset-tolerations branch April 1, 2020 01:28
dghubble added a commit that referenced this pull request Nov 14, 2020
* Add experimental `arch` variable to Fedora CoreOS AWS,
accepting amd64 (default) or arm64 to support native
arm64/aarch64 clusters or mixed/hybrid clusters with
a worker pool of arm64 workers
* Add `daemonset_tolerations` variable to cluster module
(experimental)
* Add `node_taints` variable to workers module
* Requires flannel CNI and experimental Poseidon-built
arm64 Fedora CoreOS AMIs (published to us-east-1, us-east-2,
and us-west-1)

WARN:

* Our AMIs are experimental, may be removed at any time, and
will be removed when Fedora CoreOS publishes official arm64
AMIs. Do NOT use in production

Related:

* #682
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant