Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Could not find BigQuery jobs in not US location #1088

Closed
m22r opened this issue Apr 16, 2019 · 7 comments · Fixed by #1767
Closed

Could not find BigQuery jobs in not US location #1088

m22r opened this issue Apr 16, 2019 · 7 comments · Fixed by #1767
Assignees
Labels

Comments

@m22r
Copy link

m22r commented Apr 16, 2019

I'm trying to throw a query to BigQuery's dataset in asia-northeast1, but digdag can't find job's status.
It seem to be searching jobs with format ${PROJECT_NAME}:${JOB_ID} instead of ${PROJECT_NAME}:${LOCATION}:${JOB_ID}. isn't it?

error logs

2019-04-16 21:41:43 +0900 [ERROR] (0095@[0:dataFlow]+main+test) io.digdag.core.agent.OperatorManager: Task +main+test failed.
digdag_1       | 404 Not Found
digdag_1       | {
digdag_1       |   "code" : 404,
digdag_1       |   "errors" : [ {
digdag_1       |     "domain" : "global",
digdag_1       |     "message" : "Not found: Job my-gcp-project:digdag_s0_p_my-digdag-project_w_main_t_950_a_13_6aa3ad0c-ebd2-4f78-a6b7-0c3e47957436",
digdag_1       |     "reason" : "notFound"
digdag_1       |   } ],
digdag_1       |   "message" : "Not found: Job my-gcp-project:digdag_s0_p_my-digdag-project_w_main_t_950_a_13_6aa3ad0c-ebd2-4f78-a6b7-0c3e47957436",
digdag_1       |   "status" : "NOT_FOUND"
digdag_1       | } (google json response)

main.dig

test:
   bq>: queries/test.sql
   location: asia-northeast1
   dataset: my_dataset

queries/test.sql

SELECT
  *
FROM
  `my_dataset.my_table`
@yoyama
Copy link
Contributor

yoyama commented Apr 17, 2019

@m22r Thanks for your report. Unfortunately I don't have GCP account to check your report. Please wait for a while.

@yoyama yoyama self-assigned this Apr 17, 2019
@muga muga added the bug label Apr 19, 2019
@kosukekurimoto
Copy link

I have the same problem. Is this bug likely to be resolved?

@yoyama
Copy link
Contributor

yoyama commented Feb 13, 2020

Sorry, I don't have time to handle this yet. But I think this should be solved.

@kosukekurimoto
Copy link

A temporary solution is possible by using the bq command.

+my_step:
  sh>: bq --location=asia-northeast1 query --project_id=${BIGQUERY_GCP_PROJECT} --batch --destination_table='${BIGQUERY_OPEN_DATASET}.${BIGQUERY_TABLE}' --replace=true --use_legacy_sql=false 'SELECT * FROM `${BIGQUERY_DATASET}.${BIGQUERY_TABLE}`'

@t-kazu
Copy link

t-kazu commented Feb 3, 2021

When will this problem be resolved?
Do I need to use the bq command?

@hiroyuki-sato
Copy link
Contributor

Hello, @t-kazu
AFAIK, Unfortunately, you still need to use bq command.

@toyama0919
Copy link
Contributor

Hi.

location support not yet available?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants