Cannot query data when using GCE authentication type

[pkchuyen]

Hi,

I cannot query the data from BigQuery when switch to use GCP service account.
Here is the config I set to provision Grafana Datasource:

    - name: Google BigQuery
      type: doitintl-bigquery-datasource
      access: proxy
      jsonData:
        authenticationType: gce
        defaultProject: gcp-project-sandbox
      editable: true

BQ Data is live on gcp-project-sandbox project.
When I tried to test connect to BQ from Grafana, It is able to connect But show the message about the missing project
BigQuery: Invalid project ID ''. Project IDs must contain 6-63 lowercase letters, digits, or dashes. Some project IDs also include domain name separated by a colon. IDs must start with a letter and may not end with a dash.

I tried to query data from Grafana Dashboard by including the project.dataset.tabe but unable to retrieve the data too.

Expect:

• Able to query data from BQ when using GCE as authenticationType.

Could you please point out what is the correct setup to provision Grafana BQ datasource ?

Thanks

[Tamirk]

I suggest you double-check your service account IAM configuration and verify it has access to gcp-project-sandbox.
Please share some more details and screenshot to help us troubleshoot if you don't see any issue with your configuration

[pkchuyen]

I suggest you double-check your service account IAM configuration and verify it has access to gcp-project-sandbox.
Please share some more details and screenshot to help us troubleshoot if you don't see any issue with your configuration

Thanks @Tamirk for taking a look at my issue.
I think BQ API is enabled on my GCP project and service account has sufficient permission to query the data. The only problem is it cannot pick up the default GCP project so I cannot query the data from BQ to create the graph,
By using the same service account, I exported the service account key and add it on Grafana, it works. But I cannot sort it out when tried to use authenticationType: gce

[LiorRacer]

Hi @pkchuyen,
I checked your issue by setting up an environment on GKE, here is a tutorial for it.
Please pay attention to this part of the tutorial:

Let me know if it solve your issue

[nemmyam]

Hi @LiorRacer
I'm having the same issue and the problem occurs only when connecting BigQuery.
When I added stackdriver as a data source for example the connection succeeded.
Also. notice in the picture provided by @pkchuyen where he indded followed those instructions by providing the same auth type you copied. (Default GCE Service Account)

[LiorRacer]

Hi @pkchuyen & @nemmyam
Can you both verify your default project with this command: gcloud config get-value project

[pkchuyen]

Hi @LiorRacer
The issue has been solved for me few weeks ago and I forgot to update the ticket (my apology). Actually, I needed a help from a Developer who digged into the code and showed me the correct config.

    - name: Google BigQuery
      type: doitintl-bigquery-datasource
      access: proxy
      jsonData:
        authenticationType: gce
        defaultProject: <project-name>
        processingLocation: EU
      editable: false

in my case, I am using workload Identity to load the service account credentials to auth with GCP BQ.
It set default data processing location too which enables us query the data without specify the data location, It is helpful for the Dashboard variable.

Thanks Lior for taking your time to check my issue,
Very appreciated it!

[Tamirk]

Thanks @pkchuyen for updating us... I'll close this ticket for now and if something else will come up we can open a new one.