Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: add AMI requirement note #5904

Merged
merged 1 commit into from
Mar 21, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions website/content/en/docs/concepts/nodeclasses.md
Original file line number Diff line number Diff line change
Expand Up @@ -457,6 +457,7 @@ AMIs may be specified by any AWS tag, including `Name`. Selecting by tag or by n
If `amiSelectorTerms` match more than one AMI, Karpenter will automatically determine which AMI best fits the workloads on the launched worker node under the following constraints:

* When launching nodes, Karpenter automatically determines which architecture a custom AMI is compatible with and will use images that match an instanceType's requirements.
* Note that Karpenter **cannot** detect any requirement other than architecture. If you need to specify different AMIs for different kind of nodes (e.g. accelerated GPU AMIs), you should use a separate `EC2NodeClass`.
* If multiple AMIs are found that can be used, Karpenter will choose the latest one.
* If no AMIs are found that can be used, then no nodes will be provisioned.
{{% /alert %}}
Expand Down
1 change: 1 addition & 0 deletions website/content/en/preview/concepts/nodeclasses.md
Original file line number Diff line number Diff line change
Expand Up @@ -457,6 +457,7 @@ AMIs may be specified by any AWS tag, including `Name`. Selecting by tag or by n
If `amiSelectorTerms` match more than one AMI, Karpenter will automatically determine which AMI best fits the workloads on the launched worker node under the following constraints:

* When launching nodes, Karpenter automatically determines which architecture a custom AMI is compatible with and will use images that match an instanceType's requirements.
* Note that Karpenter **cannot** detect any requirement other than architecture. If you need to specify different AMIs for different kind of nodes (e.g. accelerated GPU AMIs), you should use a separate `EC2NodeClass`.
* If multiple AMIs are found that can be used, Karpenter will choose the latest one.
* If no AMIs are found that can be used, then no nodes will be provisioned.
{{% /alert %}}
Expand Down
21 changes: 11 additions & 10 deletions website/content/en/v0.32/concepts/nodeclasses.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ spec:
# Each term in the array of subnetSelectorTerms is ORed together
# Within a single term, all conditions are ANDed
subnetSelectorTerms:
# Select on any subnet that has the "karpenter.sh/discovery: ${CLUSTER_NAME}"
# Select on any subnet that has the "karpenter.sh/discovery: ${CLUSTER_NAME}"
# AND the "environment: test" tag OR any subnet with ID "subnet-09fa4a0a8f233a921"
- tags:
karpenter.sh/discovery: "${CLUSTER_NAME}"
Expand All @@ -46,8 +46,8 @@ spec:
# Each term in the array of securityGroupSelectorTerms is ORed together
# Within a single term, all conditions are ANDed
securityGroupSelectorTerms:
# Select on any security group that has both the "karpenter.sh/discovery: ${CLUSTER_NAME}" tag
# AND the "environment: test" tag OR any security group with the "my-security-group" name
# Select on any security group that has both the "karpenter.sh/discovery: ${CLUSTER_NAME}" tag
# AND the "environment: test" tag OR any security group with the "my-security-group" name
# OR any security group with ID "sg-063d7acfb4b06c82c"
- tags:
karpenter.sh/discovery: "${CLUSTER_NAME}"
Expand All @@ -70,8 +70,8 @@ spec:
# Each term in the array of amiSelectorTerms is ORed together
# Within a single term, all conditions are ANDed
amiSelectorTerms:
# Select on any AMI that has both the "karpenter.sh/discovery: ${CLUSTER_NAME}" tag
# AND the "environment: test" tag OR any AMI with the "my-ami" name
# Select on any AMI that has both the "karpenter.sh/discovery: ${CLUSTER_NAME}" tag
# AND the "environment: test" tag OR any AMI with the "my-ami" name
# OR any AMI with ID "ami-123"
- tags:
karpenter.sh/discovery: "${CLUSTER_NAME}"
Expand Down Expand Up @@ -246,7 +246,7 @@ This selection logic is modeled as terms, where each term contains multiple cond

```yaml
subnetSelectorTerms:
# Select on any subnet that has the "karpenter.sh/discovery: ${CLUSTER_NAME}"
# Select on any subnet that has the "karpenter.sh/discovery: ${CLUSTER_NAME}"
# AND the "environment: test" tag OR any subnet with ID "subnet-09fa4a0a8f233a921"
- tags:
karpenter.sh/discovery: "${CLUSTER_NAME}"
Expand Down Expand Up @@ -313,8 +313,8 @@ This selection logic is modeled as terms, where each term contains multiple cond

```yaml
securityGroupSelectorTerms:
# Select on any security group that has both the "karpenter.sh/discovery: ${CLUSTER_NAME}" tag
# AND the "environment: test" tag OR any security group with the "my-security-group" name
# Select on any security group that has both the "karpenter.sh/discovery: ${CLUSTER_NAME}" tag
# AND the "environment: test" tag OR any security group with the "my-security-group" name
# OR any security group with ID "sg-063d7acfb4b06c82c"
- tags:
karpenter.sh/discovery: "${CLUSTER_NAME}"
Expand Down Expand Up @@ -400,8 +400,8 @@ This selection logic is modeled as terms, where each term contains multiple cond

```yaml
amiSelectorTerms:
# Select on any AMI that has both the "karpenter.sh/discovery: ${CLUSTER_NAME}" tag
# AND the "environment: test" tag OR any AMI with the "my-ami" name
# Select on any AMI that has both the "karpenter.sh/discovery: ${CLUSTER_NAME}" tag
# AND the "environment: test" tag OR any AMI with the "my-ami" name
# OR any AMI with ID "ami-123"
- tags:
karpenter.sh/discovery: "${CLUSTER_NAME}"
Expand All @@ -422,6 +422,7 @@ AMIs may be specified by any AWS tag, including `Name`. Selecting by tag or by n
If `amiSelectorTerms` match more than one AMI, Karpenter will automatically determine which AMI best fits the workloads on the launched worker node under the following constraints:

* When launching nodes, Karpenter automatically determines which architecture a custom AMI is compatible with and will use images that match an instanceType's requirements.
* Note that Karpenter **cannot** detect any requirement other than architecture. If you need to specify different AMIs for different kind of nodes (e.g. accelerated GPU AMIs), you should use a separate `EC2NodeClass`.
* If multiple AMIs are found that can be used, Karpenter will choose the latest one.
* If no AMIs are found that can be used, then no nodes will be provisioned.
{{% /alert %}}
Expand Down
21 changes: 11 additions & 10 deletions website/content/en/v0.33/concepts/nodeclasses.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ spec:
# Each term in the array of subnetSelectorTerms is ORed together
# Within a single term, all conditions are ANDed
subnetSelectorTerms:
# Select on any subnet that has the "karpenter.sh/discovery: ${CLUSTER_NAME}"
# Select on any subnet that has the "karpenter.sh/discovery: ${CLUSTER_NAME}"
# AND the "environment: test" tag OR any subnet with ID "subnet-09fa4a0a8f233a921"
- tags:
karpenter.sh/discovery: "${CLUSTER_NAME}"
Expand All @@ -46,8 +46,8 @@ spec:
# Each term in the array of securityGroupSelectorTerms is ORed together
# Within a single term, all conditions are ANDed
securityGroupSelectorTerms:
# Select on any security group that has both the "karpenter.sh/discovery: ${CLUSTER_NAME}" tag
# AND the "environment: test" tag OR any security group with the "my-security-group" name
# Select on any security group that has both the "karpenter.sh/discovery: ${CLUSTER_NAME}" tag
# AND the "environment: test" tag OR any security group with the "my-security-group" name
# OR any security group with ID "sg-063d7acfb4b06c82c"
- tags:
karpenter.sh/discovery: "${CLUSTER_NAME}"
Expand All @@ -70,8 +70,8 @@ spec:
# Each term in the array of amiSelectorTerms is ORed together
# Within a single term, all conditions are ANDed
amiSelectorTerms:
# Select on any AMI that has both the "karpenter.sh/discovery: ${CLUSTER_NAME}" tag
# AND the "environment: test" tag OR any AMI with the "my-ami" name
# Select on any AMI that has both the "karpenter.sh/discovery: ${CLUSTER_NAME}" tag
# AND the "environment: test" tag OR any AMI with the "my-ami" name
# OR any AMI with ID "ami-123"
- tags:
karpenter.sh/discovery: "${CLUSTER_NAME}"
Expand Down Expand Up @@ -246,7 +246,7 @@ This selection logic is modeled as terms, where each term contains multiple cond

```yaml
subnetSelectorTerms:
# Select on any subnet that has the "karpenter.sh/discovery: ${CLUSTER_NAME}"
# Select on any subnet that has the "karpenter.sh/discovery: ${CLUSTER_NAME}"
# AND the "environment: test" tag OR any subnet with ID "subnet-09fa4a0a8f233a921"
- tags:
karpenter.sh/discovery: "${CLUSTER_NAME}"
Expand Down Expand Up @@ -313,8 +313,8 @@ This selection logic is modeled as terms, where each term contains multiple cond

```yaml
securityGroupSelectorTerms:
# Select on any security group that has both the "karpenter.sh/discovery: ${CLUSTER_NAME}" tag
# AND the "environment: test" tag OR any security group with the "my-security-group" name
# Select on any security group that has both the "karpenter.sh/discovery: ${CLUSTER_NAME}" tag
# AND the "environment: test" tag OR any security group with the "my-security-group" name
# OR any security group with ID "sg-063d7acfb4b06c82c"
- tags:
karpenter.sh/discovery: "${CLUSTER_NAME}"
Expand Down Expand Up @@ -400,8 +400,8 @@ This selection logic is modeled as terms, where each term contains multiple cond

```yaml
amiSelectorTerms:
# Select on any AMI that has both the "karpenter.sh/discovery: ${CLUSTER_NAME}" tag
# AND the "environment: test" tag OR any AMI with the "my-ami" name
# Select on any AMI that has both the "karpenter.sh/discovery: ${CLUSTER_NAME}" tag
# AND the "environment: test" tag OR any AMI with the "my-ami" name
# OR any AMI with ID "ami-123"
- tags:
karpenter.sh/discovery: "${CLUSTER_NAME}"
Expand All @@ -422,6 +422,7 @@ AMIs may be specified by any AWS tag, including `Name`. Selecting by tag or by n
If `amiSelectorTerms` match more than one AMI, Karpenter will automatically determine which AMI best fits the workloads on the launched worker node under the following constraints:

* When launching nodes, Karpenter automatically determines which architecture a custom AMI is compatible with and will use images that match an instanceType's requirements.
* Note that Karpenter **cannot** detect any requirement other than architecture. If you need to specify different AMIs for different kind of nodes (e.g. accelerated GPU AMIs), you should use a separate `EC2NodeClass`.
* If multiple AMIs are found that can be used, Karpenter will choose the latest one.
* If no AMIs are found that can be used, then no nodes will be provisioned.
{{% /alert %}}
Expand Down
43 changes: 22 additions & 21 deletions website/content/en/v0.34/concepts/nodeclasses.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ spec:
# Each term in the array of subnetSelectorTerms is ORed together
# Within a single term, all conditions are ANDed
subnetSelectorTerms:
# Select on any subnet that has the "karpenter.sh/discovery: ${CLUSTER_NAME}"
# Select on any subnet that has the "karpenter.sh/discovery: ${CLUSTER_NAME}"
# AND the "environment: test" tag OR any subnet with ID "subnet-09fa4a0a8f233a921"
- tags:
karpenter.sh/discovery: "${CLUSTER_NAME}"
Expand All @@ -46,8 +46,8 @@ spec:
# Each term in the array of securityGroupSelectorTerms is ORed together
# Within a single term, all conditions are ANDed
securityGroupSelectorTerms:
# Select on any security group that has both the "karpenter.sh/discovery: ${CLUSTER_NAME}" tag
# AND the "environment: test" tag OR any security group with the "my-security-group" name
# Select on any security group that has both the "karpenter.sh/discovery: ${CLUSTER_NAME}" tag
# AND the "environment: test" tag OR any security group with the "my-security-group" name
# OR any security group with ID "sg-063d7acfb4b06c82c"
- tags:
karpenter.sh/discovery: "${CLUSTER_NAME}"
Expand All @@ -70,15 +70,15 @@ spec:
# Each term in the array of amiSelectorTerms is ORed together
# Within a single term, all conditions are ANDed
amiSelectorTerms:
# Select on any AMI that has both the "karpenter.sh/discovery: ${CLUSTER_NAME}" tag
# AND the "environment: test" tag OR any AMI with the "my-ami" name
# Select on any AMI that has both the "karpenter.sh/discovery: ${CLUSTER_NAME}" tag
# AND the "environment: test" tag OR any AMI with the "my-ami" name
# OR any AMI with ID "ami-123"
- tags:
karpenter.sh/discovery: "${CLUSTER_NAME}"
environment: test
- name: my-ami
- id: ami-123

# Optional, use instance-store volumes for node ephemeral-storage
instanceStorePolicy: RAID0

Expand Down Expand Up @@ -248,7 +248,7 @@ This selection logic is modeled as terms, where each term contains multiple cond

```yaml
subnetSelectorTerms:
# Select on any subnet that has the "karpenter.sh/discovery: ${CLUSTER_NAME}"
# Select on any subnet that has the "karpenter.sh/discovery: ${CLUSTER_NAME}"
# AND the "environment: test" tag OR any subnet with ID "subnet-09fa4a0a8f233a921"
- tags:
karpenter.sh/discovery: "${CLUSTER_NAME}"
Expand Down Expand Up @@ -315,8 +315,8 @@ This selection logic is modeled as terms, where each term contains multiple cond

```yaml
securityGroupSelectorTerms:
# Select on any security group that has both the "karpenter.sh/discovery: ${CLUSTER_NAME}" tag
# AND the "environment: test" tag OR any security group with the "my-security-group" name
# Select on any security group that has both the "karpenter.sh/discovery: ${CLUSTER_NAME}" tag
# AND the "environment: test" tag OR any security group with the "my-security-group" name
# OR any security group with ID "sg-063d7acfb4b06c82c"
- tags:
karpenter.sh/discovery: "${CLUSTER_NAME}"
Expand Down Expand Up @@ -402,8 +402,8 @@ This selection logic is modeled as terms, where each term contains multiple cond

```yaml
amiSelectorTerms:
# Select on any AMI that has both the "karpenter.sh/discovery: ${CLUSTER_NAME}" tag
# AND the "environment: test" tag OR any AMI with the "my-ami" name
# Select on any AMI that has both the "karpenter.sh/discovery: ${CLUSTER_NAME}" tag
# AND the "environment: test" tag OR any AMI with the "my-ami" name
# OR any AMI with ID "ami-123"
- tags:
karpenter.sh/discovery: "${CLUSTER_NAME}"
Expand All @@ -424,6 +424,7 @@ AMIs may be specified by any AWS tag, including `Name`. Selecting by tag or by n
If `amiSelectorTerms` match more than one AMI, Karpenter will automatically determine which AMI best fits the workloads on the launched worker node under the following constraints:

* When launching nodes, Karpenter automatically determines which architecture a custom AMI is compatible with and will use images that match an instanceType's requirements.
* Note that Karpenter **cannot** detect any requirement other than architecture. If you need to specify different AMIs for different kind of nodes (e.g. accelerated GPU AMIs), you should use a separate `EC2NodeClass`.
* If multiple AMIs are found that can be used, Karpenter will choose the latest one.
* If no AMIs are found that can be used, then no nodes will be provisioned.
{{% /alert %}}
Expand Down Expand Up @@ -628,25 +629,25 @@ The `instanceStorePolicy` field controls how [instance-store](https://docs.aws.a

If you intend to use these volumes for faster node ephemeral-storage, set `instanceStorePolicy` to `RAID0`:

```yaml
spec:
```yaml
spec:
instanceStorePolicy: RAID0
```

This will set the allocatable ephemeral-storage of each node to the total size of the instance-store volume(s).
This will set the allocatable ephemeral-storage of each node to the total size of the instance-store volume(s).

The disks must be formatted & mounted in a RAID0 and be the underlying filesystem for the Kubelet & Containerd. Instructions for each AMI family are listed below:
The disks must be formatted & mounted in a RAID0 and be the underlying filesystem for the Kubelet & Containerd. Instructions for each AMI family are listed below:

#### AL2
#### AL2

On AL2, Karpenter automatically configures the disks through an additional boostrap argument (`--local-disks raid0`). The device name is `/dev/md/0` and its mount point is `/mnt/k8s-disks/0`. You should ensure any additional disk setup does not interfere with these.
On AL2, Karpenter automatically configures the disks through an additional boostrap argument (`--local-disks raid0`). The device name is `/dev/md/0` and its mount point is `/mnt/k8s-disks/0`. You should ensure any additional disk setup does not interfere with these.

#### Others
#### Others

For all other AMI families, you must configure the disks yourself. Check out the [`setup-local-disks`](https://github.com/awslabs/amazon-eks-ami/blob/master/files/bin/setup-local-disks) script in [amazon-eks-ami](https://github.com/awslabs/amazon-eks-ami) to see how this is done for AL2.
For all other AMI families, you must configure the disks yourself. Check out the [`setup-local-disks`](https://github.com/awslabs/amazon-eks-ami/blob/master/files/bin/setup-local-disks) script in [amazon-eks-ami](https://github.com/awslabs/amazon-eks-ami) to see how this is done for AL2.

{{% alert title="Tip" color="secondary" %}}
Since the Kubelet & Containerd will be using the instance-store filesystem, you may consider using a more minimal root volume size.
{{% alert title="Tip" color="secondary" %}}
Since the Kubelet & Containerd will be using the instance-store filesystem, you may consider using a more minimal root volume size.
{{% /alert %}}

## spec.userData
Expand Down
1 change: 1 addition & 0 deletions website/content/en/v0.35/concepts/nodeclasses.md
Original file line number Diff line number Diff line change
Expand Up @@ -457,6 +457,7 @@ AMIs may be specified by any AWS tag, including `Name`. Selecting by tag or by n
If `amiSelectorTerms` match more than one AMI, Karpenter will automatically determine which AMI best fits the workloads on the launched worker node under the following constraints:

* When launching nodes, Karpenter automatically determines which architecture a custom AMI is compatible with and will use images that match an instanceType's requirements.
* Note that Karpenter **cannot** detect any requirement other than architecture. If you need to specify different AMIs for different kind of nodes (e.g. accelerated GPU AMIs), you should use a separate `EC2NodeClass`.
* If multiple AMIs are found that can be used, Karpenter will choose the latest one.
* If no AMIs are found that can be used, then no nodes will be provisioned.
{{% /alert %}}
Expand Down
Loading