Make operator aware of supervisor-specific runtime features

[krnowak]

It might be beneficial for users if the operator somehow exposed some information supervisor has about the service it manages. An example could be labeling pods with their election status. So at some point some pod could have a label saying that the service here is a leader. That in order could be used in a selector of a Kubernetes Service resource to expose the leader service outside the cluster.

Another thing to expose could for example be a version of the service - it would be useful for the habitat-updater.

The information can be gotten through the HTTP gateway of the supervisor, but of course not everything would be needed to be exposed as labels on Kubernetes Pods.

Also, what needs to be taken into account that it is possible for those labels to change during the lifetime of a pod and so the operator should be careful not to clobber those labels after it created them.

[surajssd]

Can we label pods differently that are managed by same StatefulSet?

[krnowak]

Heh, no idea, never tried that actually. But if this is problematic (and most likely it will be) then there is always a possibility to send three separate CRD manifests to the cluster, which will create three habitat resources, where each creates one statefulset with number of replicas being 1.