Skip to content

Commit

Permalink
Support deploying CiliumL2AnnouncementPolicies
Browse files Browse the repository at this point in the history
L2 Announcements is a feature which makes services visible and reachable on the local area network.
This feature is primarily intended for on-premises deployments within networks without BGP based routing.
  • Loading branch information
Stephan Feurer authored and DebakelOrakel committed Aug 15, 2024
1 parent 42db065 commit f03d7f7
Show file tree
Hide file tree
Showing 29 changed files with 1,507 additions and 2 deletions.
2 changes: 2 additions & 0 deletions class/cilium.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ parameters:
- input_paths:
- ${_base_directory}/component/aggregated-clusterroles.jsonnet
- ${_base_directory}/component/egress-gateway-policies.jsonnet
- ${_base_directory}/component/l2-announcement-policies.jsonnet
- ${_base_directory}/component/bgp-control-plane.jsonnet
- ${_base_directory}/component/ocp-manage-kube-proxy.jsonnet
input_type: jsonnet
Expand All @@ -51,6 +52,7 @@ parameters:
- input_paths:
- ${_base_directory}/component/aggregated-clusterroles.jsonnet
- ${_base_directory}/component/egress-gateway-policies.jsonnet
- ${_base_directory}/component/l2-announcement-policies.jsonnet
- ${_base_directory}/component/bgp-control-plane.jsonnet
- ${_base_directory}/component/ocp-manage-kube-proxy.jsonnet
input_type: jsonnet
Expand Down
6 changes: 6 additions & 0 deletions class/defaults.yml
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,8 @@ parameters:
enabled: ${cilium:egress_gateway:enabled}
bpf:
masquerade: true
l2announcements:
enabled: ${cilium:l2_announcements:enabled}
l7Proxy: ${cilium:_egressgw_l7proxy:${cilium:egress_gateway:enabled}}
prometheus:
enabled: true
Expand Down Expand Up @@ -87,6 +89,10 @@ parameters:
generate_shadow_ranges_configmap: false
egress_ip_ranges: {}

l2_announcements:
enabled: false
policies: {}

bgp:
enabled: false
peerings: {}
Expand Down
25 changes: 25 additions & 0 deletions component/l2-announcement-policies.jsonnet
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
local com = import 'lib/commodore.libjsonnet';
local kap = import 'lib/kapitan.libjsonnet';
local kube = import 'lib/kube.libjsonnet';

local inv = kap.inventory();
local params = inv.parameters.cilium;

local CiliumL2AnnouncementPolicy(name) =
kube._Object('cilium.io/v2alpha1', 'CiliumL2AnnouncementPolicy', name) {
metadata+: {
annotations+: {
'argocd.argoproj.io/sync-options': 'SkipDryRunOnMissingResource=true,Prune=false',
},
},
};

local policies = com.generateResources(
params.l2_announcements.policies,
CiliumL2AnnouncementPolicy
);

{
[if params.l2_announcements.enabled && std.length(params.l2_announcements.policies) > 0 then
'40_l2_announcement_policies']: policies,
}
81 changes: 81 additions & 0 deletions docs/modules/ROOT/pages/references/parameters.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -542,6 +542,87 @@ spec:
<2> The DaemonSet mounts the `eip-shadow-ranges` ConfigMap as a volume.
<3> The DaemonSet is scheduled using the same node selector that's used for the `IsovalentEgressGatewayPolicy` resources


== `l2_announcements`

This section allows users to configure the [Cilium L2 Announcements / L2 Aware LB] feature.

[NOTE]
====
The current implementation (and therefore examples shown here) has only been tested with Cilium EE.
Please refer to the https://docs.cilium.io/en/stable/network/egress-gateway/#example-policy[example policy in the upstream documentation] for Cilium OSS.
====

=== `l2_announcements.enabled`

[horizontal]
type:: boolean
default:: `false`

This parameter allows users to set all the configurations necessary to enable the l2 announcement policy feature.

[NOTE]
====
It is important to adjust the client rate limit when using this feature, due to increased API usage.
See https://docs.cilium.io/en/latest/network/l2-announcements/#sizing-client-rate-limit[Sizing client rate limit] for sizing guidelines.
====

[NOTE]
====
Kube Proxy replacement mode must be enabled.
====

==== Example

[source,yaml]
----
l2_announcements:
enabled: true
cilium_helm_values:
kubeProxyReplacement: true
k8sServiceHost: api-int.${openshift:baseDomain}
k8sServicePort: "6443"
k8sClientRateLimit:
qps: 35 <1>
burst: 45 <2>
----
<1> Setting the base QPS rate.
<2> The burst QPS should be slightly higher.

=== `l2_announcements.policies`

[horizontal]
type:: object
default:: `{}`

This parameter allows users to deploy `CiliumL2AnnouncementPolicy` resources.

Each key-value pair in the parameter is converted to a `CiliumL2AnnouncementPolicy` resource.
Entries can be removed by setting the value to `null`.

See https://docs.cilium.io/en/latest/network/l2-announcements/#policies[the upstream documentation] for further explanation.

==== Example

[source,yaml]
----
l2_announcements:
policies:
color_blue:
spec:
serviceSelector:
matchLabels:
color: blue
nodeSelector:
matchExpressions:
- key: node-role.kubernetes.io/control-plane
operator: DoesNotExist
interfaces:
- ^eth[0-9]+
externalIPs: true
loadBalancerIPs: true
----

== `bgp`

This section allows users to configure the https://docs.cilium.io/en/stable/network/bgp-control-plane/[Cilium BGP control plane].
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
apiVersion: v1
kind: Namespace
metadata:
annotations: {}
labels:
name: cilium
name: cilium
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
labels:
app.kubernetes.io/part-of: cilium
name: cilium
rules:
- apiGroups:
- networking.k8s.io
resources:
- networkpolicies
verbs:
- get
- list
- watch
- apiGroups:
- discovery.k8s.io
resources:
- endpointslices
verbs:
- get
- list
- watch
- apiGroups:
- ''
resources:
- namespaces
- services
- pods
- endpoints
- nodes
verbs:
- get
- list
- watch
- apiGroups:
- coordination.k8s.io
resources:
- leases
verbs:
- create
- get
- update
- list
- delete
- apiGroups:
- apiextensions.k8s.io
resources:
- customresourcedefinitions
verbs:
- list
- watch
- get
- apiGroups:
- cilium.io
resources:
- ciliumloadbalancerippools
- ciliumbgppeeringpolicies
- ciliumbgpnodeconfigs
- ciliumbgpadvertisements
- ciliumbgppeerconfigs
- ciliumclusterwideenvoyconfigs
- ciliumclusterwidenetworkpolicies
- ciliumegressgatewaypolicies
- ciliumendpoints
- ciliumendpointslices
- ciliumenvoyconfigs
- ciliumidentities
- ciliumlocalredirectpolicies
- ciliumnetworkpolicies
- ciliumnodes
- ciliumnodeconfigs
- ciliumcidrgroups
- ciliuml2announcementpolicies
- ciliumpodippools
verbs:
- list
- watch
- apiGroups:
- cilium.io
resources:
- ciliumidentities
- ciliumendpoints
- ciliumnodes
verbs:
- create
- apiGroups:
- cilium.io
resources:
- ciliumidentities
verbs:
- update
- apiGroups:
- cilium.io
resources:
- ciliumendpoints
verbs:
- delete
- get
- apiGroups:
- cilium.io
resources:
- ciliumnodes
- ciliumnodes/status
verbs:
- get
- update
- apiGroups:
- cilium.io
resources:
- ciliumnetworkpolicies/status
- ciliumclusterwidenetworkpolicies/status
- ciliumendpoints/status
- ciliumendpoints
- ciliuml2announcementpolicies/status
- ciliumbgpnodeconfigs/status
verbs:
- patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
labels:
app.kubernetes.io/part-of: cilium
name: cilium
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cilium
subjects:
- kind: ServiceAccount
name: cilium
namespace: cilium
Loading

0 comments on commit f03d7f7

Please sign in to comment.