PrivateLoadZone resources should be immutable in CRD

[bizrad]

Brief summary

The documentation in step 4 on the procedure on this page says the following about the PLZ resources

Note that currently, resources can only be configured during the creation of a Private Load Zone. You’ll have to delete and create a new Private Load Zone if you’d like to update your configuration.

Based on this the CRD should enforce immutability on these fields so that systems like flux can use the force annotation to cause the resource to be deleted and recreated on change to the field.

This can be achieved with k8s CRD transition rules

Proposed change to https://github.com/grafana/k6-operator/blob/main/charts/k6-operator/templates/crds/plz.yaml adding the x-kubernetes-validations field as shown.

                  limits:
                    additionalProperties:
                      anyOf:
                      - type: integer
                      - type: string
                      pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                      x-kubernetes-int-or-string: true
                    type: object
                    x-kubernetes-validations:
                      - rule: "self.foo == oldSelf.foo"
                        message: Requests and limits can only be set during initial object creation. Delete and recreate the object to adjust.
                  requests:
                    additionalProperties:
                      anyOf:
                      - type: integer
                      - type: string
                      pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                      x-kubernetes-int-or-string: true
                    type: object
                    x-kubernetes-validations:
                      - rule: "self.foo == oldSelf.foo"
                        message: Requests and limits can only be set during initial object creation. Delete and recreate the object to adjust.

k6-operator version or image

ghcr.io/grafana/k6-operator:controller-v0.0.13

Helm chart version (if applicable)

3.5.0

TestRun / PrivateLoadZone YAML

apiVersion: k6.io/v1alpha1
kind: PrivateLoadZone
metadata:
  name: dev
  namespace: k6-plz
  annotations:
    # if the operator makes resources fields show as immutable this will make recreation by flux work
    "kustomize.toolkit.fluxcd.io/force": "Enabled"
spec:
  token: grafana-k6-token
  resources:
    limits:
      cpu: 256m
      memory: 1024Mi

Other environment details (if applicable)

No response

Steps to reproduce the problem

Manually edit a PrivateLoadZone and see no error or warning is raised

Expected behaviour

Fields cannot effect change without object replacement are immutable.

Actual behaviour

Changing fields appears to work

[yorugac]

Hi @bizrad, thank you for opening the issue! This is good feedback for us 👍

To clarify, we are considering making resources mutable or at least more flexible so that the user can update CR and have dynamic adjustment of runner limits, without being required to re-create the resource. So basically, the docs are describing current behavior but it is not yet set in stone that this behavior won't change in future.
I guess we could still add these validation rules now and remove them in the future version as needs be 🤔

How would such a potential change look like from your usage of PLZ with Flux annotations? Probably annotations would need to be removed together with update of k6-operator?

[yorugac]

Hello @bizrad! I've written up the issue about making resources in PrivateLoadZone definition mutable separately here:
#395

The direction we'll go here is not yet decided, but I wondered if you'd have additional feedback on topic 🙂 For example, would it be beneficial to have resources mutable in your setup?