feat: add grafana plugin and dashboards

[arbreezy]

Closes #

📑 Description

This PR introduces grafana dashboards for metrics coming from controller runtime and the custom prometheus metric to measure reconciliation errors.

I believe atm this metric is already implemented out of the box by controller_runtime_reconcile_errors_total but if we introduce more controllers in the future might be useful. Eager to get your feedback on that !

✅ Checks

• My pull request adheres to the code style of this project
• My code requires changes to the documentation
• I have updated the documentation as required
• All the tests have passed

ℹ Additional Information

This PR is using the kubebuilder's grafana plugin.
I am also attaching screenshots of Grafana dashboards.

[AlexsJones]

How do we deploy this into the cluster?

[arbreezy]

that's a good question, I think there are three options which are not mutually exclusive.

1. We just provide the json dashboard files and let user copy/paste them unto their Grafana envs
2. Upload the dashboards to grafana.com
3. Create configmaps with these json files e.g
   e.g

apiVersion: v1
kind: ConfigMap
metadata:
  name: grafana-dashboards
data:
  k8sgpt.json: |
{{ .Files.Get "grafana/custom-dashboard.json" | indent 4 }}  

The only issue I have with 3. is the various different Grafana implementations ( different storage backend) where this configmap could be unused if deployed. I guess we can enable/disable it in the helm chart.

Finally kubebuilder gafana plugin helps us bootstrap the dashboards and grafana directory can be the source of truth.