Prevent updating persistentStorage on Habitat CRD

[indradhanush]

Attempting to change the size of a persistentStorage leads to the following error from the Kubernetes API:

ts=2018-07-06T12:39:03.935057362Z level=error component=controller/v1beta2 msg="Habitat could not be synced, requeueing" err="StatefulSet.apps \"example-persistent-habitat\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'template', and 'updateStrategy' are forbidden." obj=default/example-persistent-habitat

The HabitatCRD should have a check for this and not allow the update to be processed. This will prevent an infinite loop where the controller requeues the item when the update operation fails on the Kubernetes API.

[asymmetric]

The loop will not be infinite, there's a limit to how many times an object is requeued.

[indradhanush]

The link looks broken at the moment. 🤔

[asymmetric]

Oops, here's the correct link.

[iaguis]

Permalink:

habitat-operator/pkg/controller/v1beta2/controller.go

Line 148 in d23a110

queue: workqueue.NewNamedRateLimitingQueue(workqueue.DefaultControllerRateLimiter(), "Habitats"),

[krnowak]

My impression was that rate limiting is not about limiting the number of retries, but it is rather about limiting a frequency of retries (through backoff times). A documentation of NewRateLimitingQueue says:

NewRateLimitingQueue constructs a new workqueue with rateLimited queuing ability
Remember to call Forget! If you don't, you may end up tracking failures forever.

But I might be wrong. Maybe DefaultControllerRateLimiter that we use does limit retries, but I haven't read the code to check it and the docs are sh^Wless than stellar.

Also, we call Forget only when an item was processed successfully or when the item is not a Habitat object.

[asymmetric]

@krnowak You might be right. Worth looking into :)