Manage organizations

[siredmar]

Hi!

Currently i am facing the problem that i not only need to handle dashboards and data sources, but also organizations.
Is my understanding correct that this operator cannot manage organizations in any way?

Cheers
Armin

[pb82]

@siredmar That's correct, the Operator does not yet know about organizations. But that means, it will should also not overwrite ones that you create manually.

[pb82]

@siredmar Is this something you would like to see the operator support? If so, do you have more information or ideas about how this could be integrated? It looks like if we support organizations we would also have to know about users and possibly add a way of creating users via the Operator. What do you think?

[bhiravabhatla]

@pb82, Managing Orgs would be a big plus.

We were looking for ways to manage access to our dashboards using LDAP. Ideally, grafana suggests to use Teams for the same. But Teams LDAP sync is an enterprise feature.
So, we thought of using orgs to manage auth - Splitting dashboards into multiple orgs and granting access to LDAP groups at org level. Unfortunately, grafana still doesn't support provisioning organizations.
So, you cant create Orgs automatically with grafana boot up - and orgs are needed to be created before dashboards tagged to orgs are provisioned.

More details on this issue here - grafana/grafana#12119.

Using operator, we could get away from using provisioning for dashboards and use CRDs instead. But as for orgs, we would have to use some kind of prehook-job to hit grafana api and create. If you could provide a CRD for org - that would be awesome.

I am also open to raise a PR for the same, would need some help though.

[pb82]

@bhiravabhatla There is a PR for Grafana implementing Org provisioning: grafana/grafana#30578

This seems like a better approach than implementing it via API in the Operator. Once (if) this PR gets merged, you could mount a ConfigMap to provision your organizations.

The API approach would give us the advantage of dynamic Org management (through the Grafana CR). So if you want to pick this up feel free to. We are happy to provide you with help.

[bhiravabhatla]

Hi @pb82, Went through the code. As of now controller is utilizing the provisioning provided by grafana for datasources, dashboards etc by creating/updating corresponding configMap. If we would have to implement Org Provisioning with current grafana - only way I see is to use grafana API - That would need grafana being up and running before we provision.

This would mean deviating from existing strategy. I guess we could wait for above PR to be merged before implementing this in operator - Thoughts?

[pb82]

@bhiravabhatla why would the API approach deviate from the existing strategy? The dashboard controller currently already uses the Grafana API and we have a client that you can use: https://github.com/integr8ly/grafana-operator/blob/master/controllers/grafanadashboard/grafana_client.go

It needs some extra methods for org management.

[bhiravabhatla]

@pb82 - I missed this. Will go through.

went through - Yep, I could add more methods the client interface. Does it make sense to take it out grafanadashboard package and have its own package. As it wont be specific to dashboard anymore.

[pb82]

@bhiravabhatla If we add organizations to the Grafana CR then it makes sense to move that client code out of the dashboard package. It can have it's own package called grafanaclient or something similar.

[pb82]

We're happy to accept a PR for this, but its not something on our roadmap at the moment.

[bhiravabhatla]

Sorry for late update - I have started working on this, could not find time to finish - will try to find sometime next weekend. Will keep this issue updated

[Voigtus]

Since I'm currently thinking about implementing this, I invested some brain power into this.

I see two possible ways. Either we add the Org to the Grafana CR or create a new CRD.
Pros and Cons as follow

Adding to Grafana CR

PROS

• Orgs are part of a single Grafana instance. Therefore they should be added to the Grafana CR.
• One less CRD that needs to be maintained and if users need to be added to an Org this could be done in another CRD i.e. GrafanaUser.
• No issue regarding "timing", since the orgs could be applied instantly after the deployment is ready.
• URL for API-calls is already present

CONS

• If there is a name change of an Org, it can not be updated, but pre-change-Org must be deleted and new one added afterwards.

Creating new CRD

PROS

• Users can be added to the Org in the CR.
• Rename is doable
• IF the Orgs get new functionality it could be added easier

CONS

• Needs to be applied after the deployment is done.
• API-url needs to be provided to the CR

If anyone got some feedback, please let me know.
I will x-post to the slack channel later today to grab some attention regarding this topic.

[bhiravabhatla]

I could not spend time on this at all. Apologies for the same. Thanks @Voigtus for renewing this thread.

Happy to see this getting traction. Personally I feel adding it to Grafana CR makes more sense.

[kmarquardsen]

Hi @Voigtus was there a discussion around this in slack? Is it already being worked on or currently an open item for contribution?

[NissesSenap]

Hi all there are a number of people talking about the need for organizations and there is a few issues around it.
There have been some work around this but I think most of it have stopped.

I think a good start to get this done is to create a design document around CR one or more CR:s that needs to be created/updated.

This could be done as a normal PR and stored under documentation/design_proposal/organization or something like that.
I think managing organizations should include thinking of how to manage:

• Organization
• Team
• User

https://grafana.com/docs/grafana/latest/http_api/

We also need to consider how to solve this when the grafana-operator supports multiple instances.

When implementing this I don't think it should be a single PR, I would even prefer it being split up in to minor PR:s for a easier overview that way many people can also help out with it. But I would like to see someone take charge of the design proposal so the rest of the community can come with feedback on how the API should look.
As all of us know changing API:s and at the same time being backwards compatible is a pain.
Just because you create the initial design document don't mean that you have to implement the feature.

No I will ping a bunch of people that I know have been interested in this before: @kmarquardsen @siredmar @bhiravabhatla @Voigtus @DeanBrunt
If anyone else feels up for the task please go for it, it's not my intention to exclude anyone.

I hope that you think this is a good idea and that some feels up for the task.
As always feel free to reach out on the kubernetes slack.

[NissesSenap]

So I have tried to think of good design solution on how to implement multi organizational support to a single grafana instance. I'm currently leaning on not supporting multiple organization in a single grafana instance.

I have tried to explain my reasoning why and I would love to get your feedback on this: #615

[NissesSenap]

Sorry for not providing any update on this issue.

Our current feeling is that organizations isn't something that should be used inside grafana. We have spoken some with the grafana team about this topic and they agree and that is why organizations isn't supported in there cloud offering.

From our point of view the solution is to use multiple grafana instances. We know that this feature currently isn't supported for the operator and we are currently working hard on version 5 to support multi namespace mutli grafana instances.

Sorry for the bad news.
We hope that you can see our point and that you just like us is looking forward to version 5 of the operator.