EV-4694 Make resource requests/limits configurable for KB, prometheus, Alert Manager

[vara2504]

Description

Kibana:

kibana override the kibana resource and its pod template.

kibana spec

kubectl patch logstorage tigera-secure  --type=merge --patch='{"spec": {"kibana":{"spec": {"template": {"spec": {"containers":[{"name":"kibana","resources":{"limits":{"cpu":"808m", "memory":"808Mi"},"requests":{"cpu":"80m", "memory":"80Mi"}}}]}}}}}}'


vara@vara:~/bzprofiles/Clusters/tc_res$ k get deployment.apps/tigera-secure-kb -n tigera-kibana -o yaml|grep -A 7 -B 1 resources
          timeoutSeconds: 5
        resources:
          limits:
            cpu: 808m
            memory: 808Mi
          requests:
            cpu: 80m
            memory: 80Mi
        securityContext:
--
        name: elastic-internal-init-config
        resources:
          limits:
            cpu: 100m
            memory: 50Mi
          requests:
            cpu: 100m
            memory: 50Mi
        securityContext:

ECKOperator

kubectl patch logstorage tigera-secure  --type=merge --patch='{"spec": {"eckOperatorStatefulSet":{"spec": {"template": {"spec": {"containers":[{"name":"manager","resources":{"limits":{"cpu":"708m", "memory":"708Mi"},"requests":{"cpu":"107m", "memory":"107Mi"}}}]}}}}}}'
vara@vara:~/bzprofiles/Clusters/tc_res$ k get statefulset.apps/elastic-operator -n tigera-eck-operator -o yaml|grep -A 7 -B 1 resources
        name: manager
        resources:
          limits:
            cpu: 708m
            memory: 708Mi
          requests:
            cpu: 107m
            memory: 107Mi
        securityContext:

when not configured, it defaults to below values
        resources:
          limits:
            cpu: "1"
            memory: 512Mi
          requests:
            cpu: 100m
            memory: 512Mi

ECKOperatorStatefulSet takes precedence when configured:
vara@vara:~/bzprofiles/Clusters/tc_res$ k get logstorage -o yaml
apiVersion: v1
items:
- apiVersion: operator.tigera.io/v1
  kind: LogStorage
  metadata:
    creationTimestamp: "2024-03-04T18:40:40Z"
    finalizers:
    - tigera.io/eck-cleanup
    generation: 13
    name: tigera-secure
    resourceVersion: "855055"
    uid: 16e2f0b6-dc00-4888-a2ab-d062830e42d2
  spec:
    componentResources:
    - componentName: ECKOperator
      resourceRequirements:
        limits:
          memory: 512Mi
        requests:
          memory: 512Mi
    eckOperatorStatefulSet:
      spec:
        template:
          spec:
            containers:
            - name: manager
              resources:
                limits:
                  cpu: 708m
                  memory: 708Mi
                requests:
                  cpu: 107m
                  memory: 107Mi

vara@vara:~/bzprofiles/Clusters/tc_res$ k get statefulset.apps/elastic-operator -n tigera-eck-operator -o yaml|grep -A 7 -B 1 resources
        name: manager
        resources:
          limits:
            cpu: 708m
            memory: 708Mi
          requests:
            cpu: 107m
            memory: 107Mi
        securityContext:

Prometheus has defaults set for init-config-reloader, config-reloader
It allows configuring the resources for "prometheus" container via Resource param in Spec.
Any Additional container injected via Prometheus's Container can be configured (for now it supports authn-proxy)

Updates prometheus injected containers.

vara@vara:~/bzprofiles/Clusters/tc_res$ kubectl patch monitor tigera-secure  --type=merge --patch='{"spec": {"prometheus":{"spec": {"commonPrometheusFields": {"containers":[{"name":"authn-proxy","resources":{"limits":{"cpu":"150m", "memory":"500Mi"},"requests":{"cpu":"20m", "memory":"50Mi"}}}]}}}}}'
monitor.operator.tigera.io/tigera-secure patched
vara@vara:~/bzprofiles/Clusters/tc_res$ k get monitor -o yaml
apiVersion: v1
items:
- apiVersion: operator.tigera.io/v1
  kind: Monitor
  metadata:
    creationTimestamp: "2024-03-04T18:40:40Z"
    generation: 7
    name: tigera-secure
    resourceVersion: "833816"
    uid: 4ca71b10-e285-4bfa-bc00-329cd4fc5827
  spec:
    prometheus:
      spec:
        commonPrometheusFields:
          containers:
          - name: authn-proxy
            resources:
              limits:
                cpu: 150m
                memory: 500Mi
              requests:
                cpu: 20m
                memory: 50Mi

Update prometheus resources and therefore its container such as prometheus

kubectl patch monitor tigera-secure  --type=merge --patch='{"spec": {"prometheus":{"spec": {"commonPrometheusFields": {"resources":{"limits":{"cpu":"350m", "memory":"550Mi"},"requests":{"cpu":"30m", "memory":"80Mi"}}}}}}}'


vara@vara:~/bzprofiles/Clusters/tc_res$ k get monitor -o yaml
apiVersion: v1
items:
- apiVersion: operator.tigera.io/v1
  kind: Monitor
  metadata:
    creationTimestamp: "2024-03-04T18:40:40Z"
    generation: 9
    name: tigera-secure
    resourceVersion: "836990"
    uid: 4ca71b10-e285-4bfa-bc00-329cd4fc5827
  spec:
    prometheus:
      spec:
        commonPrometheusFields:
          containers:
          - name: authn-proxy
            resources:
              limits:
                cpu: 550m
                memory: 550Mi
              requests:
                cpu: 55m
                memory: 55Mi
          resources:
            limits:
              cpu: 350m
              memory: 550Mi
            requests:
              cpu: 30m
              memory: 80Mi

Update alertmanager resource:

vara@vara:~/bzprofiles/Clusters/tc_res$  kubectl patch monitor tigera-secure  --type=merge --patch='{"spec": {"alertManager": {"spec": {"resources":{"limits":{"cpu":"450m", "memory":"450Mi"},"requests":{"cpu":"40m", "memory":"40Mi"}}}}}}'
monitor.operator.tigera.io/tigera-secure patched
vara@vara:~/bzprofiles/Clusters/tc_res$ k get monitor -o yaml
apiVersion: v1
items:
- apiVersion: operator.tigera.io/v1
  kind: Monitor
  metadata:
    creationTimestamp: "2024-03-04T18:40:40Z"
    generation: 11
    name: tigera-secure
    resourceVersion: "844419"
    uid: 4ca71b10-e285-4bfa-bc00-329cd4fc5827
  spec:
    alertManager:
      spec:
        resources:
          limits:
            cpu: 450m
            memory: 450Mi
          requests:
            cpu: 40m
            memory: 40Mi
    prometheus:
      spec:
        commonPrometheusFields:
          containers:
          - name: authn-proxy
            resources:
              limits:
                cpu: 550m
                memory: 550Mi
              requests:
                cpu: 55m
                memory: 55Mi
          resources:
            limits:
              cpu: 350m
              memory: 550Mi
            requests:
              cpu: 30m
              memory: 80Mi

For PR author

• Tests for change.
• If changing pkg/apis/, run make gen-files
• If changing versions, run make gen-versions

For PR reviewers

A note for code reviewers - all pull requests must have the following:

• Milestone set according to targeted release.
• Appropriate labels:
  • kind/bug if this is a bugfix.
  • kind/enhancement if this is a a new feature.
  • enterprise if this PR applies to Calico Enterprise only.

[vara2504]

Placed ApplyPrometheusOverrides func inside components to take advantage of the merge container functionlity of components.

[vara2504]

Placed ApplyKibanaOverrides func inside components to take advantage of the existing replicated pod resources and applyReplicatedPodResourceOverrides func. Since kibana spec has Pod Template which can be overridden in the same way.

[rene-dekker]

lgtm