Reduce memory footprint for secrets

[timuthy]

What would you like to be added:
We've been observing an immense amount of memory usage if the source cluster contains many secrets.

Example:

Secret count (not referenced by certificates): ~ 3200
Volume per Secret: ~ 1 MB
Memory needed for Cert-Management component: ~ 6.7 GB

Please check if the memory footprint can be reduced in such scenarios.

Suggestions:

• Use informer with tweaked list options, e.g. only consider special labelled secerts
• Use Metadata only informer (ref ✨ metadata-only watches kubernetes-sigs/controller-runtime#1174)
• Bypass informer cache if possible

[timuthy]

/cc @mandelsoft @timebertt