OSS OnCall fails to set-up connection to Grafana Cloud OnCall: "Invalid token"

[sirvincent]

What went wrong?

What happened:

• Inserting Grafana cloud OnCall API token into Grafana OnCall OSS (docker) fails with message: "Invalid token"
  

What did you expect to happen:

• Connection between Grafana cloud & local OnCall

How do we reproduce it?

1. Open Grafana Cloud OnCall
2. Create API Token
3. Copy API Token into local Grafana OSS that runs Grafana OnCall OSS in docker
4. See error "Invalid Token"

Grafana OnCall Version

v1.11.3

Product Area

API

Grafana OnCall Platform?

Docker

User's Browser?

firefox 131.0.2

Anything else to add?

For the complete picture:
We run locally grafana OSS version 11.2.2 but use our cloud dashboard API key.
Which communicates to Grafana OnCall OSS (v1.11.3) within it's docker container.
I do not see anything strange, no errors or similar, in the grafana logs, docker celery and engine logs.

[youve]

I reported a similar issue on the forum a while ago. I have screenshots of how to reproduce this issue: https://community.grafana.com/t/trying-to-connect-selfhosted-grafana-to-cloud-oncall/129985/6

In our case we are using plugin version 1.11.3, but we also had this issue with 1.9.22 and with some earlier version. Our grafana version 10.4.1 in self-hosted kubernetes.

[bpedersen2]

This sounds like related to #5100

[sirvincent]

We have managed to fix this problem by specifying grafana cloud on call api URL in the environment:
GRAFANA_CLOUD_ONCALL_API_URL: https://oncall-prod-eu-west-0.grafana.net/oncall
Specifically that key is now part of our x-environment in the hobby docker compose yaml file.

Our cloud grafana is in europe not US.
We see that in base.py if not specified it evaluates to the US version URL.
Do note we need to click twice connect, in the connect to cloud settings, before OnCall registers it.

We have a new consecutive problem that the IRM app sparingly scans (registers) the QR code, for both the OSS & cloud version. However that is a separate problem so I will close this issue.

[pcockwell]

We have managed to fix this problem by specifying grafana cloud on call api URL in the environment: GRAFANA_CLOUD_ONCALL_API_URL: https://oncall-prod-eu-west-0.grafana.net/oncall Specifically that key is now part of our x-environment in the hobby docker compose yaml file.

Our cloud grafana is in europe not US. We see that in base.py if not specified it evaluates to the US version URL. Do note we need to click twice connect, in the connect to cloud settings, before OnCall registers it.

We have a new consecutive problem that the IRM app sparingly scans (registers) the QR code, for both the OSS & cloud version. However that is a separate problem so I will close this issue.

I am using helm to deploy an OSS copy of Grafana and Grafana OnCall separately. @sirvincent Where did you set that environment variable? On Grafana? On the Oncall OSS deployment? Both? Elsewhere?

[sirvincent]

I set it in the docker-compose.yml file, we are using the hobby environment.

x-environment: &oncall-environment
  GRAFANA_CLOUD_ONCALL_API_URL: https://oncall-prod-eu-west-0.grafana.net/oncall

I do not know how to set environment variables for the production helm install.
I do see this _env.tpl file which seem to contain some environment variables.

[chuchynz]

I do not know how to set environment variables for the production helm install. I do see this _env.tpl file which seem to contain some environment variables.

You can add it to the env: section of values.yaml. Example:

  env:
    GRAFANA_CLOUD_ONCALL_API_URL: https://oncall-prod-eu-west-0.grafana.net/oncall

[alext-extracellular]

Thanks sirvincent for posting your solution. I think this should be mentioned somewhere in the readme, as it caught me out too.

[youve]

Thanks sirvincent for posting your solution. I think this should be mentioned somewhere in the readme, as it caught me out too.

I agree, I opened a pull request for it, because I was blocked on setting up OnCall for several months over this issue

[alext-extracellular]

@youve Brilliant! thank you for doing the work I did not have the time/willpower to do :)