Cannot create multiple Grafana in the same namespace.

[dulltz]

I tried to create multiple Grafana CRs in the same namesapce.
However, the generated resource names are duplicated since they are hard-coded such as grafana-service or grafana-admin-credentials.
As a result of it, I failed to create multiple Grafana CR on the same namesapce.

I think we should support multiple Grafana in the same namespace.
If grafana-operator builds resource names as ${grafana-name}-${resource-name}, this issue would be solved.

[dulltz]

Moreover, it seems that grafana-operator cannot detect Grafana CRs in namespaces except for grafana-operator's namespace. (ref)

As a result of them, grafana-operator can deploy single grafana only, is it true?

[rverma-jm]

Facing issue cause of this, it would be nice if we can support a different namespace for grafana. Its alright(even preferred) to place data sources in same namespace as in grafana.

[hubeadmin]

@dulltz @rverma-jm There's some planned work involving the ability for Grafana's to be managed by operators from different namespaces coming up soon, Not sure of the timeframe though so I can't give a definite "When".

[dulltz]

@HubertStefanski
Thank you for your reply.
By the way, where is the future development plan of grafana-operator being mainly discussed?
I would like to see it if possible.

[hubeadmin]

@dulltz Most of this planning for now is done internally as part of the development process for the Integreatly-Operator, as needs arise for functionality we delegate to work on the grafana-operator, we are actively ensuring that we take in a few issues for the grafana-operator in each sprint to keep it moving forward.
As of now I'm not sure if there are any external documents/places where work is being planned, but perhaps this is something to consider? @pb82

[pb82]

@dulltz @HubertStefanski Yes, currently the development of this operator is mainly discussed via Github issues and in our internal chat rooms. We can consider creating a google group for it.

[dulltz]

@HubertStefanski @pb82
Thank you for your responce. I got the workflow of Integreatly-Operator a little.

We can consider creating a google group for it.

I'm concerned that running a Google Group will be too costly for the core members.
If the core members are not troubled at the moment, I think the current GitHub Issue-based development is fine.

[NissesSenap]

Any update on this?

[pb82]

@NissesSenap @dulltz We are working on publishing a roadmap, but it will still take some time. Multi namespace support is considered, but probably not one of the top priorities from our side (although we certainly will accept community pull requests for this).

[d-kuro]

I'd like to address this issue.

As for running multiple Grafana in the same namespace, I believe that this can be easily solved by not creating resources with fixed names and prefixing them with the .metadata.name of the Grafana resource.

e.g.:

  func GrafanaDeployment(cr *v1alpha1.Grafana, configHash, dsHash string) *v1.Deployment {
	  return &v1.Deployment{
		  ObjectMeta: v12.ObjectMeta{
-			  Name:      GrafanaDeploymentName,
+			  Name:      fmt.Sprintf("%s-%s", cr.Name, GrafanaDeploymentName),
			  Namespace: cr.Namespace,
		  },
		  Spec: getDeploymentSpec(cr, nil, configHash, "", dsHash),
	  }
  }

https://github.com/integr8ly/grafana-operator/blob/v3.9.0/pkg/controller/model/grafanaDeployment.go#L561

However, there is a problem with the GrafanaDataSource resource.

The current logic looks like this (If I’m wrong, please comment):

1. Grafana datasource config file is located in a ConfigMap named grafana-datasources.
2. grafana-datasources ConfgMap is created by Grafana controller.
   • https://github.com/integr8ly/grafana-operator/blob/v3.9.0/pkg/controller/model/grafanaDatasources.go#L10
   • https://github.com/integr8ly/grafana-operator/blob/v3.9.0/pkg/controller/grafana/grafana_reconciler.go#L192-L195
3. In the reconciliation loop for GrafanaDataSource, refer to the ConfigMap with the fixed name grafana-datasources and write the Grafana datasource config file.
   • https://github.com/integr8ly/grafana-operator/blob/v3.9.0/pkg/controller/common/dataSourcesState.go#L51-L69
   • https://github.com/integr8ly/grafana-operator/blob/v3.9.0/pkg/controller/grafanadatasource/datasource_pipeline.go#L55-L61
   • https://github.com/integr8ly/grafana-operator/blob/v3.9.0/pkg/controller/grafanadatasource/datasource_controller.go#L207-L208

If you prefix grafana-datasource ConfigMap with the name of Grafana object, you will not be able to find ConfigMap in the reconciliation loop of GrafanaDataSource with the logic as it is.

Also, the GrafanaDashboard and GrafanaDataSource resources share the state of the Grafana controller in the ControllerState struct.
This struct is currently designed to manage only one Grafana, so multiple Grafanas cannot be managed.

• https://github.com/integr8ly/grafana-operator/blob/v3.9.0/pkg/controller/common/controllerState.go#L7-L13
• https://github.com/integr8ly/grafana-operator/blob/v3.9.0/pkg/controller/grafanadashboard/dashboard_controller.go#L118
• https://github.com/integr8ly/grafana-operator/blob/v3.9.0/pkg/controller/grafanadatasource/datasource_controller.go#L115

I think grafana-operator is currently used by many companies, so we have to be careful about breaking changes.

---

I consider this to be a difficult and complex issue.
I think the solution with minimal changes is that the ReconcileGrafanaDashboard and ReconcileGrafanaDataSource have multiple ControllerState. (Alternatively, you may want to stop keeping the state in a structure and use the API to get it every time. Can consider storing the information needed to construct the ControllerState structure in the Grafana Object status.)
And add a new field to the Grafana CRD.

  dashboardLabelSelectors:
    type: array
    items:
      type: object
      description: Label selector or match expressions
+ datasourceLabelSelectors:
+   type: array
+   items:
+     type: object
+     description: Label selector or match expressions

GrafanaDashboard and GrafanaDataSource controllers refer to the labelselector of the multiple Grafana objects they hold, determine the match, and execute the process. (This can be may inefficient)

Do you have any other good ideas for this?

[d-kuro]

We had a conversation in the Kubernetes Slack thread:
https://kubernetes.slack.com/archives/C019A1KTYKC/p1616416376015000

[d-kuro]

I have created a PoC for architectural changes in grafana-operator to solve this problem:
https://github.com/d-kuro/grafana-operator/pull/1

[pb82]

@d-kuro thanks a lot! We'd like to evaluate your PR as soon as we can. This seems to address both issues. Would you mind trying a rebase on latest master on your branch?

[hubeadmin]

Resolved in #919