Helm operator: namespace scope resources with Helm keep annotations are being deleted by garbage collection

[mikeshng]

Bug Report

What did you do?

I created a sample Helm operator, then modified one of the namespace scope resources in the template to contain:

metadata:
  annotations:
    "helm.sh/resource-policy": keep

see "Tell Helm Not To Uninstall a Resource" in: https://helm.sh/docs/howto/charts_tips_and_tricks for more details.

After applying the Helm sample CR and then proceed to delete it, the resource with the keep annotation is also deleted from the cluster.

When I tried the same annotation on a cluster scope resource, that resource is not deleted so I think the issue is due to parent child garbage collection with namespace scope resources.

What did you expect to see?

If possible, it would be nice to have the namespace resources that are annotated with the keep to still exists after the CR is deleted. Or the website/doc should mention that keep annotation is not supported for namespace scope resources.

What did you see instead? Under which circumstances?

namespace scope resources with Helm keep annotations are being deleted when the Helm operator CR is removed.

Environment

Operator type:
/language helm

Kubernetes cluster type:
kind

$ operator-sdk version

operator-sdk version: "v1.3.0-7-g8c31ab8f", commit: "8c31ab8fdbbe572a94a10c8f0dff33bbb53beb38", kubernetes version: "v1.19.4", go version: "go1.14.1", GOOS: "linux", GOARCH: "amd64"

$ kubectl version

Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.2-0-g52c56ce", GitCommit:"b66f2d3a6893be729f1b8660309a59c6e0b69196", GitTreeState:"clean", BuildDate:"2020-08-10T04:49:09Z", GoVersion:"go1.13.4", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.0", GitCommit:"70132b0f130acc0bed193d9ba59dd186f0e634cf", GitTreeState:"clean", BuildDate:"2020-01-14T00:09:19Z", GoVersion:"go1.13.4", Compiler:"gc", Platform:"linux/amd64"}

Possible Solution

• Before removing the CR finalizer https://github.com/operator-framework/operator-sdk/blob/v1.3.0/internal/helm/controller/reconcile.go#L137 disconnect the child resources with Helm keep annotations from the parent so that garbage collection will not delete them.
• Or just update the website/doc to mention that Helm keep annotation is not supported for namespace scope resources

[joelanford]

@mikeshng I think this is happening because the helm-operator injects an owner reference into all the resources in the manifest. So when the CR is deleted, first a helm uninstall happens (which wouldn't delete these resources), then the CR finalizer is removed and the CR is deleted, then Kubernetes GCs all owned resources.

I think the fix would be to NOT inject owner references into resources that have this annotation.