Skip to content

Commit

Permalink
graduate ServiceIPStaticSubrange to GA
Browse files Browse the repository at this point in the history
  • Loading branch information
aojea committed Nov 18, 2022
1 parent 89e8f36 commit 6f7421b
Show file tree
Hide file tree
Showing 4 changed files with 144 additions and 15 deletions.
3 changes: 1 addition & 2 deletions content/en/blog/_posts/2022-05-03-kubernetes-release-1.24.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,8 +84,7 @@ that enables the caller of a function to control all aspects of logging (output
### Avoiding Collisions in IP allocation to Services

Kubernetes 1.24 introduces a new opt-in feature that allows you to
[soft-reserve a range for static IP address assignments](/docs/concepts/services-networking/service/#service-ip-static-sub-range)
to Services.
soft-reserve a range for static IP address assignments to Services.
With the manual enablement of this feature, the cluster will prefer automatic assignment from
the pool of Service IP addresses, thereby reducing the risk of collision.

Expand Down
139 changes: 139 additions & 0 deletions content/en/docs/concepts/services-networking/cluster-ip-allocation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,139 @@
---
reviewers:
- sftim
- thockin
title: Service ClusterIP allocation
content_type: concept
weight: 120
---


<!-- overview -->

In Kubernetes, [Services](/docs/concepts/services-networking/service/) are an abstract way to expose
an application running on a set of Pods. Services
can have a cluster-scoped virtual IP address (using a Service of `type: ClusterIP`).
Clients can connect using that virtual IP address, and Kubernetes then load-balances traffic to that
Service across the different backing Pods.

<!-- body -->

## How Service ClusterIPs are allocated?

A Service `ClusterIP` can be assigned:

_dynamically_
: the cluster's control plane automatically picks a free IP address from within the configured IP range for `type: ClusterIP` Services.

_statically_
: you specify an IP address of your choice, from within the configured IP range for Services.

Across your whole cluster, every Service `ClusterIP` must be unique.
Trying to create a Service with a specific `ClusterIP` that has already
been allocated will return an error.

## Why do you need to reserve Service Cluster IPs?

Sometimes you may want to have Services running in well-known IP addresses, so other components and
users in the cluster can use them.

The best example is the DNS Service for the cluster. Some Kubernetes installers assign the 10th address from
the Service IP range to the DNS service. Assuming you configured your cluster with Service IP range
10.96.0.0/16 and you want your DNS Service IP to be 10.96.0.10, you'd have to create a Service like
this:

```yaml
apiVersion: v1
kind: Service
metadata:
labels:
k8s-app: kube-dns
kubernetes.io/cluster-service: "true"
kubernetes.io/name: CoreDNS
name: kube-dns
namespace: kube-system
spec:
clusterIP: 10.96.0.10
ports:
- name: dns
port: 53
protocol: UDP
targetPort: 53
- name: dns-tcp
port: 53
protocol: TCP
targetPort: 53
selector:
k8s-app: kube-dns
type: ClusterIP
```
but as it was explained before, the IP address 10.96.0.10 has not been reserved; if other Services are created
before or in parallel with dynamic allocation, there is a chance they can allocate this IP, hence,
you will not be able to create the DNS Service because it will fail with a conflict error.
## How can you avoid Service ClusterIP conflicts? {#avoid-ClusterIP-conflict}
The allocation strategy implemented in Kubernetes to allocate ClusterIPs to Services reduces the
risk of collision.
The `ClusterIP` range is divided, based on the formula `min(max(16, cidrSize / 16), 256)`,
described as _never less than 16 or more than 256 with a graduated step between them_.

Dynamic IP assignment uses the upper band by default, once this has been exhausted it will
use the lower range. This will allow users to use static allocations on the lower band with a low
risk of collision.

Examples:

#### Service IP CIDR block: 10.96.0.0/24

Range Size: 2<sup>8</sup> - 2 = 254
Band Offset: `min(max(16, 256/16), 256)` = `min(16, 256)` = 16
Static band start: 10.96.0.1
Static band end: 10.96.0.16
Range end: 10.96.0.254

{{< mermaid >}}
pie showData
title 10.96.0.0/24
"Static" : 16
"Dynamic" : 238
{{< /mermaid >}}

#### Service IP CIDR block: 10.96.0.0/20

Range Size: 2<sup>12</sup> - 2 = 4094
Band Offset: `min(max(16, 4096/16), 256)` = `min(256, 256)` = 256
Static band start: 10.96.0.1
Static band end: 10.96.1.0
Range end: 10.96.15.254

{{< mermaid >}}
pie showData
title 10.96.0.0/20
"Static" : 256
"Dynamic" : 3838
{{< /mermaid >}}

#### Service IP CIDR block: 10.96.0.0/16

Range Size: 2<sup>16</sup> - 2 = 65534
Band Offset: `min(max(16, 65536/16), 256)` = `min(4096, 256)` = 256
Static band start: 10.96.0.1
Static band ends: 10.96.1.0
Range end: 10.96.255.254

{{< mermaid >}}
pie showData
title 10.96.0.0/16
"Static" : 256
"Dynamic" : 65278
{{< /mermaid >}}

## {{% heading "whatsnext" %}}

* Read about [Service External Traffic Policy](/docs/tasks/access-application-cluster/create-external-load-balancer/#preserving-the-client-source-ip)
* Read about [Connecting Applications with Services](/docs/concepts/services-networking/connect-applications-service/)
* Read about [Services](/docs/concepts/services-networking/service/)

12 changes: 1 addition & 11 deletions content/en/docs/concepts/services-networking/service.md
Original file line number Diff line number Diff line change
Expand Up @@ -1398,17 +1398,7 @@ in-memory locking). Kubernetes also uses controllers to check for invalid
assignments (eg due to administrator intervention) and for cleaning up allocated
IP addresses that are no longer used by any Services.

#### IP address ranges for `type: ClusterIP` Services {#service-ip-static-sub-range}

{{< feature-state for_k8s_version="v1.25" state="beta" >}}
However, there is a problem with this `ClusterIP` allocation strategy, because a user
can also [choose their own address for the service](#choosing-your-own-ip-address).
This could result in a conflict if the internal allocator selects the same IP address
for another Service.

The `ServiceIPStaticSubrange`
[feature gate](/docs/reference/command-line-tools-reference/feature-gates/) is enabled by default in v1.25
and later, using an allocation strategy that divides the `ClusterIP` range into two bands, based on
Services are using an [allocation strategy](/docs/concepts/services-networking/cluster-ip-allocation/) that divides the `ClusterIP` range into two bands, based on
the size of the configured `service-cluster-ip-range` by using the following formula
`min(max(16, cidrSize / 16), 256)`, described as _never less than 16 or more than 256,
with a graduated step function between them_. Dynamic IP allocations will be preferentially
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -172,8 +172,6 @@ For a reference to old feature gates that are removed, please refer to
| `SeccompDefault` | `true` | Beta | 1.25 | |
| `ServerSideFieldValidation` | `false` | Alpha | 1.23 | 1.24 |
| `ServerSideFieldValidation` | `true` | Beta | 1.25 | |
| `ServiceIPStaticSubrange` | `false` | Alpha | 1.24 | 1.24 |
| `ServiceIPStaticSubrange` | `true` | Beta | 1.25 | |
| `SizeMemoryBackedVolumes` | `false` | Alpha | 1.20 | 1.21 |
| `SizeMemoryBackedVolumes` | `true` | Beta | 1.22 | |
| `StatefulSetAutoDeletePVC` | `false` | Alpha | 1.22 | |
Expand Down Expand Up @@ -310,6 +308,9 @@ For a reference to old feature gates that are removed, please refer to
| `ServiceInternalTrafficPolicy` | `false` | Alpha | 1.21 | 1.21 |
| `ServiceInternalTrafficPolicy` | `true` | Beta | 1.22 | 1.25 |
| `ServiceInternalTrafficPolicy` | `true` | GA | 1.26 | - |
| `ServiceIPStaticSubrange` | `false` | Alpha | 1.24 | 1.24 |
| `ServiceIPStaticSubrange` | `true` | Beta | 1.25 | 1.25 |
| `ServiceIPStaticSubrange` | `true` | GA | 1.26 | - |
| `ServiceLBNodePortControl` | `false` | Alpha | 1.20 | 1.21 |
| `ServiceLBNodePortControl` | `true` | Beta | 1.22 | 1.23 |
| `ServiceLBNodePortControl` | `true` | GA | 1.24 | - |
Expand Down

0 comments on commit 6f7421b

Please sign in to comment.