Skip to content

Commit

Permalink
README: Add note about cluster-autoscaler not supporting multiple AZs
Browse files Browse the repository at this point in the history
  • Loading branch information
mgalgs committed Mar 23, 2019
1 parent 2a1189c commit 05f03d5
Showing 1 changed file with 38 additions and 1 deletion.
39 changes: 38 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,7 @@ To use a 3-5 node Auto Scaling Group, run:
eksctl create cluster --name=cluster-5 --nodes-min=3 --nodes-max=5
```

> NOTE: You will still need to install and configure autoscaling. See the "Enable Autoscaling" section below.
> NOTE: You will still need to install and configure autoscaling. See the "Enable Autoscaling" section below. Also note that depending on your workloads you might need to use a separate nodegroup for each AZ. See [Zone-aware Autoscaling](#zone-aware-autoscaling) below for more info.
To use 30 `c4.xlarge` nodes and prevent updating current context in `~/.kube/config`, run:

Expand Down Expand Up @@ -410,6 +410,43 @@ and `k8s.io/cluster-autoscaler/<clusterName>` tags, so nodegroup discovery shoul

[cluster autoscaler]: https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/cloudprovider/aws/README.md

#### Zone-aware Autoscaling

If your workloads are zone-specific you'll need to create separate nodegroups for each zone. This is because the `cluster-autoscaler` assumes that all nodes in a group are exactly equivalent. So, for example, if a scale-up event is triggered by a pod which needs a zone-specific PVC (e.g. an EBS volume), the new node might get scheduled in the wrong AZ and the pod will fail to start.

You won't need a separate nodegroup for each AZ if your environment meets the following criteria:

- The AZRebalance scaling process is [suspended](https://docs.aws.amazon.com/autoscaling/ec2/userguide/as-suspend-resume-processes.html).
- No zone-specific storage requirements.
- No podAffinity with topology other than host.
- No nodeAffinity on zone label.
- Never scale any zone to 0.

(Read more [here](https://github.com/kubernetes/autoscaler/pull/1802#issuecomment-474295002) and [here](https://github.com/weaveworks/eksctl/pull/647#issuecomment-474698054).)

If you meet all of the above requirements (and possibly others) then you should be safe with a single nodegroup which spans multiple AZs. Otherwise you'll want to create separate, single-AZ nodegroups:

BEFORE:

```yaml
nodeGroups:
- name: ng1-public
instanceType: m5.xlarge
# availabilityZones: ["eu-west-2a", "eu-west-2b"]
```

AFTER:

```yaml
nodeGroups:
- name: ng1-public-2a
instanceType: m5.xlarge
availabilityZones: ["eu-west-2a"]
- name: ng1-public-2b
instanceType: m5.xlarge
availabilityZones: ["eu-west-2b"]
```

### VPC Networking

By default, `eksctl create cluster` will build a dedicated VPC, in order to avoid interference with any existing resources for a
Expand Down

0 comments on commit 05f03d5

Please sign in to comment.